2023-03-08 11:53:28,867 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-08 11:53:29,139 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8 2023-03-08 11:53:29,155 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-08 11:53:29,155 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/cluster_7d11bbd1-2ad2-65f1-e179-3d37987b50d1, deleteOnExit=true 2023-03-08 11:53:29,156 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-08 11:53:29,156 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/test.cache.data in system properties and HBase conf 2023-03-08 11:53:29,156 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/hadoop.tmp.dir in system properties and HBase conf 2023-03-08 11:53:29,157 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/hadoop.log.dir in system properties and HBase conf 2023-03-08 11:53:29,157 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-08 11:53:29,158 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-08 11:53:29,158 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-08 11:53:29,267 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-08 11:53:29,681 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-08 11:53:29,685 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-08 11:53:29,685 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-08 11:53:29,686 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-08 11:53:29,686 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-08 11:53:29,686 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-08 11:53:29,686 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-08 11:53:29,687 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-08 11:53:29,687 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-08 11:53:29,687 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-08 11:53:29,688 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/nfs.dump.dir in system properties and HBase conf 2023-03-08 11:53:29,688 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/java.io.tmpdir in system properties and HBase conf 2023-03-08 11:53:29,688 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-08 11:53:29,688 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-08 11:53:29,689 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-08 11:53:30,254 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-08 11:53:30,258 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-08 11:53:30,551 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-08 11:53:30,741 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-08 11:53:30,758 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-08 11:53:30,796 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-08 11:53:30,827 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/java.io.tmpdir/Jetty_localhost_45191_hdfs____.xmqq9a/webapp 2023-03-08 11:53:30,962 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45191 2023-03-08 11:53:30,978 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-08 11:53:30,978 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-08 11:53:31,500 WARN [Listener at localhost/33137] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-08 11:53:31,591 WARN [Listener at localhost/33137] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-08 11:53:31,619 WARN [Listener at localhost/33137] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-08 11:53:31,626 INFO [Listener at localhost/33137] log.Slf4jLog(67): jetty-6.1.26 2023-03-08 11:53:31,638 INFO [Listener at localhost/33137] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/java.io.tmpdir/Jetty_localhost_34739_datanode____.89w9bx/webapp 2023-03-08 11:53:31,737 INFO [Listener at localhost/33137] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34739 2023-03-08 11:53:32,146 WARN [Listener at localhost/36211] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-08 11:53:32,734 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xde10be649648c686: Processing first storage report for DS-2c309f17-2367-4961-97cc-9b36e6a50c3b from datanode 8ded10a7-69e0-452e-8b64-90e5bfab4eb6 2023-03-08 11:53:32,736 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xde10be649648c686: from storage DS-2c309f17-2367-4961-97cc-9b36e6a50c3b node DatanodeRegistration(127.0.0.1:45059, datanodeUuid=8ded10a7-69e0-452e-8b64-90e5bfab4eb6, infoPort=37827, infoSecurePort=0, ipcPort=36211, storageInfo=lv=-57;cid=testClusterID;nsid=846450660;c=1678276410337), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-03-08 11:53:32,736 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xde10be649648c686: Processing first storage report for DS-b8dd49cb-bff6-4805-aa65-3050ba82372c from datanode 8ded10a7-69e0-452e-8b64-90e5bfab4eb6 2023-03-08 11:53:32,736 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xde10be649648c686: from storage DS-b8dd49cb-bff6-4805-aa65-3050ba82372c node DatanodeRegistration(127.0.0.1:45059, datanodeUuid=8ded10a7-69e0-452e-8b64-90e5bfab4eb6, infoPort=37827, infoSecurePort=0, ipcPort=36211, storageInfo=lv=-57;cid=testClusterID;nsid=846450660;c=1678276410337), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-08 11:53:32,805 DEBUG [Listener at localhost/36211] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8 2023-03-08 11:53:32,882 INFO [Listener at localhost/36211] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/cluster_7d11bbd1-2ad2-65f1-e179-3d37987b50d1/zookeeper_0, clientPort=51863, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/cluster_7d11bbd1-2ad2-65f1-e179-3d37987b50d1/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/cluster_7d11bbd1-2ad2-65f1-e179-3d37987b50d1/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-08 11:53:32,896 INFO [Listener at localhost/36211] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51863 2023-03-08 11:53:32,907 INFO [Listener at localhost/36211] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 11:53:32,910 INFO [Listener at localhost/36211] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 11:53:33,589 INFO [Listener at localhost/36211] util.FSUtils(471): Created version file at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c with version=8 2023-03-08 11:53:33,590 INFO [Listener at localhost/36211] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/hbase-staging 2023-03-08 11:53:33,905 INFO [Listener at localhost/36211] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-08 11:53:34,385 INFO [Listener at localhost/36211] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-03-08 11:53:34,419 INFO [Listener at localhost/36211] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 11:53:34,420 INFO [Listener at localhost/36211] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-08 11:53:34,420 INFO [Listener at localhost/36211] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-08 11:53:34,420 INFO [Listener at localhost/36211] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 11:53:34,420 INFO [Listener at localhost/36211] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-08 11:53:34,570 INFO [Listener at localhost/36211] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-08 11:53:34,649 DEBUG [Listener at localhost/36211] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-08 11:53:34,754 INFO [Listener at localhost/36211] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46487 2023-03-08 11:53:34,765 INFO [Listener at localhost/36211] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 11:53:34,767 INFO [Listener at localhost/36211] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 11:53:34,788 INFO [Listener at localhost/36211] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46487 connecting to ZooKeeper ensemble=127.0.0.1:51863 2023-03-08 11:53:34,830 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:464870x0, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-08 11:53:34,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:46487-0x100c3f9af850000 connected 2023-03-08 11:53:34,858 DEBUG [Listener at localhost/36211] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 11:53:34,858 DEBUG [Listener at localhost/36211] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 11:53:34,862 DEBUG [Listener at localhost/36211] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-08 11:53:34,869 DEBUG [Listener at localhost/36211] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46487 2023-03-08 11:53:34,870 DEBUG [Listener at localhost/36211] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46487 2023-03-08 11:53:34,870 DEBUG [Listener at localhost/36211] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46487 2023-03-08 11:53:34,871 DEBUG [Listener at localhost/36211] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46487 2023-03-08 11:53:34,871 DEBUG [Listener at localhost/36211] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46487 2023-03-08 11:53:34,877 INFO [Listener at localhost/36211] master.HMaster(444): hbase.rootdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c, hbase.cluster.distributed=false 2023-03-08 11:53:34,947 INFO [Listener at localhost/36211] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-03-08 11:53:34,947 INFO [Listener at localhost/36211] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 11:53:34,947 INFO [Listener at localhost/36211] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-08 11:53:34,947 INFO [Listener at localhost/36211] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-08 11:53:34,947 INFO [Listener at localhost/36211] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 11:53:34,948 INFO [Listener at localhost/36211] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-08 11:53:34,952 INFO [Listener at localhost/36211] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-08 11:53:34,956 INFO [Listener at localhost/36211] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45551 2023-03-08 11:53:34,958 INFO [Listener at localhost/36211] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-08 11:53:34,964 DEBUG [Listener at localhost/36211] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-08 11:53:34,965 INFO [Listener at localhost/36211] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 11:53:34,968 INFO [Listener at localhost/36211] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 11:53:34,970 INFO [Listener at localhost/36211] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45551 connecting to ZooKeeper ensemble=127.0.0.1:51863 2023-03-08 11:53:34,974 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:455510x0, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-08 11:53:34,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:45551-0x100c3f9af850001 connected 2023-03-08 11:53:34,976 DEBUG [Listener at localhost/36211] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 11:53:34,977 DEBUG [Listener at localhost/36211] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 11:53:34,978 DEBUG [Listener at localhost/36211] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-08 11:53:34,982 DEBUG [Listener at localhost/36211] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45551 2023-03-08 11:53:34,983 DEBUG [Listener at localhost/36211] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45551 2023-03-08 11:53:34,983 DEBUG [Listener at localhost/36211] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45551 2023-03-08 11:53:34,984 DEBUG [Listener at localhost/36211] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45551 2023-03-08 11:53:34,984 DEBUG [Listener at localhost/36211] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45551 2023-03-08 11:53:34,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,46487,1678276413741 2023-03-08 11:53:34,995 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-08 11:53:34,997 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,46487,1678276413741 2023-03-08 11:53:35,016 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-08 11:53:35,016 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-08 11:53:35,016 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 11:53:35,017 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-08 11:53:35,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,46487,1678276413741 from backup master directory 2023-03-08 11:53:35,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-08 11:53:35,025 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,46487,1678276413741 2023-03-08 11:53:35,025 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-08 11:53:35,026 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-08 11:53:35,026 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,46487,1678276413741 2023-03-08 11:53:35,029 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-08 11:53:35,030 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-08 11:53:35,115 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/hbase.id with ID: ab4f5600-b9d6-40dc-92e5-f097a8013066 2023-03-08 11:53:35,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 11:53:35,171 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 11:53:35,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x62758e02 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:53:35,245 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@60cd8dea, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:53:35,271 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-08 11:53:35,273 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-08 11:53:35,291 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-08 11:53:35,291 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-08 11:53:35,293 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-08 11:53:35,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-08 11:53:35,299 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 11:53:35,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/data/master/store-tmp 2023-03-08 11:53:35,365 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:35,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-08 11:53:35,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 11:53:35,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 11:53:35,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-08 11:53:35,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 11:53:35,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 11:53:35,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 11:53:35,368 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/WALs/jenkins-hbase4.apache.org,46487,1678276413741 2023-03-08 11:53:35,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46487%2C1678276413741, suffix=, logDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/WALs/jenkins-hbase4.apache.org,46487,1678276413741, archiveDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/oldWALs, maxLogs=10 2023-03-08 11:53:35,450 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45059,DS-2c309f17-2367-4961-97cc-9b36e6a50c3b,DISK] 2023-03-08 11:53:35,461 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-08 11:53:35,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/WALs/jenkins-hbase4.apache.org,46487,1678276413741/jenkins-hbase4.apache.org%2C46487%2C1678276413741.1678276415400 2023-03-08 11:53:35,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45059,DS-2c309f17-2367-4961-97cc-9b36e6a50c3b,DISK]] 2023-03-08 11:53:35,528 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:53:35,528 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:35,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 11:53:35,534 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 11:53:35,606 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-08 11:53:35,613 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-08 11:53:35,640 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-08 11:53:35,654 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:35,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-08 11:53:35,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-08 11:53:35,682 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 11:53:35,687 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:53:35,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62262599, jitterRate=-0.07221497595310211}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:53:35,689 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 11:53:35,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-08 11:53:35,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-08 11:53:35,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-08 11:53:35,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-08 11:53:35,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-08 11:53:35,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 34 msec 2023-03-08 11:53:35,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-08 11:53:35,779 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-08 11:53:35,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-08 11:53:35,809 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-08 11:53:35,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-08 11:53:35,815 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-08 11:53:35,820 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-08 11:53:35,824 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-08 11:53:35,828 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 11:53:35,830 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-08 11:53:35,830 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-08 11:53:35,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-08 11:53:35,855 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-08 11:53:35,855 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-08 11:53:35,855 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 11:53:35,855 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,46487,1678276413741, sessionid=0x100c3f9af850000, setting cluster-up flag (Was=false) 2023-03-08 11:53:35,870 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 11:53:35,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-08 11:53:35,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46487,1678276413741 2023-03-08 11:53:35,884 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 11:53:35,889 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-08 11:53:35,891 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46487,1678276413741 2023-03-08 11:53:35,893 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/.tmp 2023-03-08 11:53:35,988 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(952): ClusterId : ab4f5600-b9d6-40dc-92e5-f097a8013066 2023-03-08 11:53:35,993 DEBUG [RS:0;jenkins-hbase4:45551] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-08 11:53:35,998 DEBUG [RS:0;jenkins-hbase4:45551] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-08 11:53:35,999 DEBUG [RS:0;jenkins-hbase4:45551] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-08 11:53:36,003 DEBUG [RS:0;jenkins-hbase4:45551] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-08 11:53:36,004 DEBUG [RS:0;jenkins-hbase4:45551] zookeeper.ReadOnlyZKClient(139): Connect 0x59cbeeba to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:53:36,013 DEBUG [RS:0;jenkins-hbase4:45551] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@570c1bf3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:53:36,014 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-08 11:53:36,014 DEBUG [RS:0;jenkins-hbase4:45551] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@533385ca, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-08 11:53:36,027 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 11:53:36,027 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 11:53:36,027 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 11:53:36,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 11:53:36,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-03-08 11:53:36,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-08 11:53:36,029 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1678276446037 2023-03-08 11:53:36,039 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-08 11:53:36,041 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-08 11:53:36,041 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-08 11:53:36,043 DEBUG [RS:0;jenkins-hbase4:45551] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:45551 2023-03-08 11:53:36,047 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-08 11:53:36,049 INFO [RS:0;jenkins-hbase4:45551] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-08 11:53:36,049 INFO [RS:0;jenkins-hbase4:45551] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-08 11:53:36,049 DEBUG [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1023): About to register with Master. 2023-03-08 11:53:36,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-08 11:53:36,053 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46487,1678276413741 with isa=jenkins-hbase4.apache.org/172.31.14.131:45551, startcode=1678276414946 2023-03-08 11:53:36,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-08 11:53:36,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-08 11:53:36,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-08 11:53:36,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-08 11:53:36,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-08 11:53:36,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-08 11:53:36,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-08 11:53:36,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-08 11:53:36,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-08 11:53:36,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678276416071,5,FailOnTimeoutGroup] 2023-03-08 11:53:36,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678276416071,5,FailOnTimeoutGroup] 2023-03-08 11:53:36,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-08 11:53:36,073 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,073 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,079 DEBUG [RS:0;jenkins-hbase4:45551] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-08 11:53:36,085 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-08 11:53:36,086 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-08 11:53:36,087 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c 2023-03-08 11:53:36,102 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:36,105 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-08 11:53:36,109 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/info 2023-03-08 11:53:36,110 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-08 11:53:36,112 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:36,112 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-08 11:53:36,116 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/rep_barrier 2023-03-08 11:53:36,117 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-08 11:53:36,118 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:36,119 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-08 11:53:36,121 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/table 2023-03-08 11:53:36,123 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-08 11:53:36,124 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:36,127 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740 2023-03-08 11:53:36,128 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740 2023-03-08 11:53:36,133 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-08 11:53:36,136 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-08 11:53:36,141 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:53:36,142 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71273405, jitterRate=0.06205649673938751}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-08 11:53:36,142 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-08 11:53:36,142 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-08 11:53:36,143 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-08 11:53:36,143 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-08 11:53:36,143 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-08 11:53:36,143 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-08 11:53:36,144 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-08 11:53:36,145 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-08 11:53:36,147 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60953, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-08 11:53:36,156 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-08 11:53:36,156 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-08 11:53:36,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:36,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-08 11:53:36,179 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-08 11:53:36,182 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-08 11:53:36,186 DEBUG [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c 2023-03-08 11:53:36,186 DEBUG [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:33137 2023-03-08 11:53:36,186 DEBUG [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-08 11:53:36,191 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 11:53:36,191 DEBUG [RS:0;jenkins-hbase4:45551] zookeeper.ZKUtil(162): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:36,192 WARN [RS:0;jenkins-hbase4:45551] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-08 11:53:36,192 INFO [RS:0;jenkins-hbase4:45551] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 11:53:36,193 DEBUG [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1947): logDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/WALs/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:36,193 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45551,1678276414946] 2023-03-08 11:53:36,202 DEBUG [RS:0;jenkins-hbase4:45551] zookeeper.ZKUtil(162): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:36,214 DEBUG [RS:0;jenkins-hbase4:45551] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-08 11:53:36,222 INFO [RS:0;jenkins-hbase4:45551] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-08 11:53:36,245 INFO [RS:0;jenkins-hbase4:45551] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-08 11:53:36,250 INFO [RS:0;jenkins-hbase4:45551] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:53:36,251 INFO [RS:0;jenkins-hbase4:45551] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,252 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-08 11:53:36,259 INFO [RS:0;jenkins-hbase4:45551] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,260 DEBUG [RS:0;jenkins-hbase4:45551] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,260 DEBUG [RS:0;jenkins-hbase4:45551] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,260 DEBUG [RS:0;jenkins-hbase4:45551] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,260 DEBUG [RS:0;jenkins-hbase4:45551] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,260 DEBUG [RS:0;jenkins-hbase4:45551] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,261 DEBUG [RS:0;jenkins-hbase4:45551] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-08 11:53:36,261 DEBUG [RS:0;jenkins-hbase4:45551] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,261 DEBUG [RS:0;jenkins-hbase4:45551] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,261 DEBUG [RS:0;jenkins-hbase4:45551] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,261 DEBUG [RS:0;jenkins-hbase4:45551] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 11:53:36,262 INFO [RS:0;jenkins-hbase4:45551] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,262 INFO [RS:0;jenkins-hbase4:45551] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,263 INFO [RS:0;jenkins-hbase4:45551] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,280 INFO [RS:0;jenkins-hbase4:45551] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-08 11:53:36,283 INFO [RS:0;jenkins-hbase4:45551] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45551,1678276414946-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,300 INFO [RS:0;jenkins-hbase4:45551] regionserver.Replication(203): jenkins-hbase4.apache.org,45551,1678276414946 started 2023-03-08 11:53:36,300 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,45551,1678276414946, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45551, sessionid=0x100c3f9af850001 2023-03-08 11:53:36,300 DEBUG [RS:0;jenkins-hbase4:45551] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-08 11:53:36,300 DEBUG [RS:0;jenkins-hbase4:45551] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:36,300 DEBUG [RS:0;jenkins-hbase4:45551] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45551,1678276414946' 2023-03-08 11:53:36,301 DEBUG [RS:0;jenkins-hbase4:45551] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:36,301 DEBUG [RS:0;jenkins-hbase4:45551] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:36,302 DEBUG [RS:0;jenkins-hbase4:45551] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-08 11:53:36,302 DEBUG [RS:0;jenkins-hbase4:45551] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-08 11:53:36,302 DEBUG [RS:0;jenkins-hbase4:45551] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:36,302 DEBUG [RS:0;jenkins-hbase4:45551] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45551,1678276414946' 2023-03-08 11:53:36,302 DEBUG [RS:0;jenkins-hbase4:45551] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 11:53:36,303 DEBUG [RS:0;jenkins-hbase4:45551] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 11:53:36,303 DEBUG [RS:0;jenkins-hbase4:45551] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-08 11:53:36,303 INFO [RS:0;jenkins-hbase4:45551] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-08 11:53:36,303 INFO [RS:0;jenkins-hbase4:45551] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-08 11:53:36,334 DEBUG [jenkins-hbase4:46487] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-08 11:53:36,337 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45551,1678276414946, state=OPENING 2023-03-08 11:53:36,344 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-08 11:53:36,346 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 11:53:36,347 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-08 11:53:36,350 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:53:36,418 INFO [RS:0;jenkins-hbase4:45551] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45551%2C1678276414946, suffix=, logDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/WALs/jenkins-hbase4.apache.org,45551,1678276414946, archiveDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/oldWALs, maxLogs=32 2023-03-08 11:53:36,437 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45059,DS-2c309f17-2367-4961-97cc-9b36e6a50c3b,DISK] 2023-03-08 11:53:36,441 INFO [RS:0;jenkins-hbase4:45551] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/WALs/jenkins-hbase4.apache.org,45551,1678276414946/jenkins-hbase4.apache.org%2C45551%2C1678276414946.1678276416421 2023-03-08 11:53:36,442 DEBUG [RS:0;jenkins-hbase4:45551] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45059,DS-2c309f17-2367-4961-97cc-9b36e6a50c3b,DISK]] 2023-03-08 11:53:36,542 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:36,545 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 11:53:36,549 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58314, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 11:53:36,562 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-08 11:53:36,563 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 11:53:36,568 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45551%2C1678276414946.meta, suffix=.meta, logDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/WALs/jenkins-hbase4.apache.org,45551,1678276414946, archiveDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/oldWALs, maxLogs=32 2023-03-08 11:53:36,589 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45059,DS-2c309f17-2367-4961-97cc-9b36e6a50c3b,DISK] 2023-03-08 11:53:36,593 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/WALs/jenkins-hbase4.apache.org,45551,1678276414946/jenkins-hbase4.apache.org%2C45551%2C1678276414946.meta.1678276416570.meta 2023-03-08 11:53:36,593 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45059,DS-2c309f17-2367-4961-97cc-9b36e6a50c3b,DISK]] 2023-03-08 11:53:36,593 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:53:36,595 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-08 11:53:36,615 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-08 11:53:36,621 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-08 11:53:36,628 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-08 11:53:36,628 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:36,628 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-08 11:53:36,628 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-08 11:53:36,632 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-08 11:53:36,634 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/info 2023-03-08 11:53:36,634 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/info 2023-03-08 11:53:36,635 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-08 11:53:36,636 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:36,637 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-08 11:53:36,638 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/rep_barrier 2023-03-08 11:53:36,639 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/rep_barrier 2023-03-08 11:53:36,640 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-08 11:53:36,641 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:36,641 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-08 11:53:36,643 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/table 2023-03-08 11:53:36,643 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/table 2023-03-08 11:53:36,644 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-08 11:53:36,645 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:36,648 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740 2023-03-08 11:53:36,652 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740 2023-03-08 11:53:36,656 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-08 11:53:36,659 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-08 11:53:36,661 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58973095, jitterRate=-0.12123240530490875}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-08 11:53:36,661 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-08 11:53:36,672 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1678276416532 2023-03-08 11:53:36,693 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-08 11:53:36,693 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-08 11:53:36,694 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45551,1678276414946, state=OPEN 2023-03-08 11:53:36,696 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-08 11:53:36,696 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-08 11:53:36,701 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-08 11:53:36,701 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45551,1678276414946 in 346 msec 2023-03-08 11:53:36,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-08 11:53:36,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 537 msec 2023-03-08 11:53:36,712 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 774 msec 2023-03-08 11:53:36,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1678276416713, completionTime=-1 2023-03-08 11:53:36,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-08 11:53:36,714 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-03-08 11:53:36,800 DEBUG [hconnection-0x126a5d82-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:53:36,804 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58318, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:53:36,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-03-08 11:53:36,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1678276476828 2023-03-08 11:53:36,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1678276536828 2023-03-08 11:53:36,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 114 msec 2023-03-08 11:53:36,855 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46487,1678276413741-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,856 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46487,1678276413741-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,856 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46487,1678276413741-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:46487, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-08 11:53:36,873 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-08 11:53:36,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-08 11:53:36,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-08 11:53:36,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-08 11:53:36,901 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:53:36,905 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:53:36,929 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:53:36,931 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540 empty. 2023-03-08 11:53:36,932 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:53:36,932 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-08 11:53:37,365 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-08 11:53:37,368 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => f1ce88c61365d5c7b6e6a24b8a887540, NAME => 'hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:53:37,783 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:37,783 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing f1ce88c61365d5c7b6e6a24b8a887540, disabling compactions & flushes 2023-03-08 11:53:37,783 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:53:37,783 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:53:37,783 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. after waiting 0 ms 2023-03-08 11:53:37,783 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:53:37,783 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:53:37,783 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for f1ce88c61365d5c7b6e6a24b8a887540: 2023-03-08 11:53:37,788 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:53:37,803 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678276417791"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276417791"}]},"ts":"1678276417791"} 2023-03-08 11:53:37,831 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 11:53:37,833 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:53:37,838 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276417833"}]},"ts":"1678276417833"} 2023-03-08 11:53:37,841 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-08 11:53:37,850 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f1ce88c61365d5c7b6e6a24b8a887540, ASSIGN}] 2023-03-08 11:53:37,852 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f1ce88c61365d5c7b6e6a24b8a887540, ASSIGN 2023-03-08 11:53:37,854 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=f1ce88c61365d5c7b6e6a24b8a887540, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:53:38,005 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f1ce88c61365d5c7b6e6a24b8a887540, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:38,005 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678276418005"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276418005"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276418005"}]},"ts":"1678276418005"} 2023-03-08 11:53:38,011 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure f1ce88c61365d5c7b6e6a24b8a887540, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:53:38,170 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:53:38,171 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f1ce88c61365d5c7b6e6a24b8a887540, NAME => 'hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:53:38,172 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:53:38,172 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:38,172 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:53:38,172 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:53:38,174 INFO [StoreOpener-f1ce88c61365d5c7b6e6a24b8a887540-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:53:38,176 DEBUG [StoreOpener-f1ce88c61365d5c7b6e6a24b8a887540-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540/info 2023-03-08 11:53:38,177 DEBUG [StoreOpener-f1ce88c61365d5c7b6e6a24b8a887540-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540/info 2023-03-08 11:53:38,177 INFO [StoreOpener-f1ce88c61365d5c7b6e6a24b8a887540-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1ce88c61365d5c7b6e6a24b8a887540 columnFamilyName info 2023-03-08 11:53:38,178 INFO [StoreOpener-f1ce88c61365d5c7b6e6a24b8a887540-1] regionserver.HStore(310): Store=f1ce88c61365d5c7b6e6a24b8a887540/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:38,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:53:38,180 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:53:38,186 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:53:38,190 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:53:38,191 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f1ce88c61365d5c7b6e6a24b8a887540; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59724845, jitterRate=-0.11003045737743378}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:53:38,191 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f1ce88c61365d5c7b6e6a24b8a887540: 2023-03-08 11:53:38,193 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540., pid=6, masterSystemTime=1678276418164 2023-03-08 11:53:38,198 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:53:38,198 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:53:38,200 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f1ce88c61365d5c7b6e6a24b8a887540, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:38,200 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678276418199"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276418199"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276418199"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276418199"}]},"ts":"1678276418199"} 2023-03-08 11:53:38,211 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-08 11:53:38,211 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure f1ce88c61365d5c7b6e6a24b8a887540, server=jenkins-hbase4.apache.org,45551,1678276414946 in 194 msec 2023-03-08 11:53:38,216 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-08 11:53:38,216 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=f1ce88c61365d5c7b6e6a24b8a887540, ASSIGN in 361 msec 2023-03-08 11:53:38,218 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:53:38,219 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276418218"}]},"ts":"1678276418218"} 2023-03-08 11:53:38,223 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-08 11:53:38,228 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:53:38,230 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 1.3460 sec 2023-03-08 11:53:38,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-08 11:53:38,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-08 11:53:38,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 11:53:38,337 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-08 11:53:38,355 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 11:53:38,362 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 33 msec 2023-03-08 11:53:38,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-08 11:53:38,385 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 11:53:38,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 19 msec 2023-03-08 11:53:38,398 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-08 11:53:38,402 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-08 11:53:38,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.376sec 2023-03-08 11:53:38,404 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-08 11:53:38,406 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-08 11:53:38,406 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-08 11:53:38,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46487,1678276413741-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-08 11:53:38,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46487,1678276413741-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-08 11:53:38,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-08 11:53:38,495 DEBUG [Listener at localhost/36211] zookeeper.ReadOnlyZKClient(139): Connect 0x5759ba4d to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:53:38,499 DEBUG [Listener at localhost/36211] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@449b853a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:53:38,513 DEBUG [hconnection-0x684e302b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:53:38,525 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58332, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:53:38,535 INFO [Listener at localhost/36211] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,46487,1678276413741 2023-03-08 11:53:38,536 DEBUG [Listener at localhost/36211] zookeeper.ReadOnlyZKClient(139): Connect 0x47652242 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:53:38,540 DEBUG [Listener at localhost/36211] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@143ae868, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:53:38,540 INFO [Listener at localhost/36211] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 11:53:38,562 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=209, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=292, ProcessCount=170, AvailableMemoryMB=7721 2023-03-08 11:53:38,572 DEBUG [Listener at localhost/36211] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:53:38,576 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56332, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:53:38,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 11:53:38,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:38,591 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:53:38,593 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:53:38,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-08 11:53:38,596 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:38,597 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1 empty. 2023-03-08 11:53:38,599 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:38,599 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-08 11:53:38,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 11:53:38,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 11:53:38,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 11:53:39,016 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-08 11:53:39,018 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 09169a8d7fcfaa680f26277a4640bff1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:53:39,031 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:39,032 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 09169a8d7fcfaa680f26277a4640bff1, disabling compactions & flushes 2023-03-08 11:53:39,032 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:39,032 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:39,032 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. after waiting 0 ms 2023-03-08 11:53:39,032 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:39,032 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:39,032 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:39,037 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:53:39,039 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678276419039"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276419039"}]},"ts":"1678276419039"} 2023-03-08 11:53:39,043 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 11:53:39,044 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:53:39,045 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276419044"}]},"ts":"1678276419044"} 2023-03-08 11:53:39,047 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-08 11:53:39,058 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=09169a8d7fcfaa680f26277a4640bff1, ASSIGN}] 2023-03-08 11:53:39,060 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=09169a8d7fcfaa680f26277a4640bff1, ASSIGN 2023-03-08 11:53:39,063 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=09169a8d7fcfaa680f26277a4640bff1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:53:39,214 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=09169a8d7fcfaa680f26277a4640bff1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:39,214 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678276419214"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276419214"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276419214"}]},"ts":"1678276419214"} 2023-03-08 11:53:39,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 11:53:39,217 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 09169a8d7fcfaa680f26277a4640bff1, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:53:39,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:39,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 09169a8d7fcfaa680f26277a4640bff1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:53:39,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:39,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:39,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:39,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:39,380 INFO [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:39,382 DEBUG [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1 2023-03-08 11:53:39,382 DEBUG [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1 2023-03-08 11:53:39,385 INFO [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09169a8d7fcfaa680f26277a4640bff1 columnFamilyName family1 2023-03-08 11:53:39,388 INFO [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] regionserver.HStore(310): Store=09169a8d7fcfaa680f26277a4640bff1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:39,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:39,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:39,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:39,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:53:39,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 09169a8d7fcfaa680f26277a4640bff1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74465284, jitterRate=0.10961920022964478}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:53:39,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:39,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1., pid=11, masterSystemTime=1678276419370 2023-03-08 11:53:39,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:39,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:39,407 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=09169a8d7fcfaa680f26277a4640bff1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:39,407 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678276419407"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276419407"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276419407"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276419407"}]},"ts":"1678276419407"} 2023-03-08 11:53:39,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-08 11:53:39,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 09169a8d7fcfaa680f26277a4640bff1, server=jenkins-hbase4.apache.org,45551,1678276414946 in 193 msec 2023-03-08 11:53:39,416 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-08 11:53:39,416 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=09169a8d7fcfaa680f26277a4640bff1, ASSIGN in 355 msec 2023-03-08 11:53:39,417 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:53:39,417 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276419417"}]},"ts":"1678276419417"} 2023-03-08 11:53:39,420 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-08 11:53:39,423 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:53:39,425 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 836 msec 2023-03-08 11:53:39,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 11:53:39,718 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-08 11:53:39,739 DEBUG [hconnection-0x137d7f40-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:53:39,742 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58336, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:53:39,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:39,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:39,887 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:39,887 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:39,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 11:53:39,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 11:53:39,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:39,889 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:39,890 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:39,890 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:39,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:39,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:39,890 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:39,891 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:39,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:39,891 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:39,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:39,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:39,892 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:39,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:39,894 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 11:53:39,894 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:39,895 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 11:53:39,895 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:39,895 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:39,895 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:39,896 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. started... 2023-03-08 11:53:39,897 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 09169a8d7fcfaa680f26277a4640bff1 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 11:53:40,438 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113034155f03a9014456b44d4079a48c520f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113034155f03a9014456b44d4079a48c520f 2023-03-08 11:53:40,859 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/e355568068724fa787b45dbf02f4b995 2023-03-08 11:53:40,869 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/e355568068724fa787b45dbf02f4b995 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e355568068724fa787b45dbf02f4b995 2023-03-08 11:53:40,877 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e355568068724fa787b45dbf02f4b995, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-08 11:53:40,879 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 09169a8d7fcfaa680f26277a4640bff1 in 982ms, sequenceid=5, compaction requested=false 2023-03-08 11:53:40,881 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:40,883 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:40,883 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:40,883 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:40,884 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:40,884 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,884 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 11:53:40,884 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 11:53:40,887 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,887 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:40,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:40,887 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,887 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:40,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:40,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:40,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:40,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 11:53:40,889 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 11:53:40,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28a50ad5[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:40,890 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,891 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,891 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 11:53:40,891 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 11:53:40,891 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,891 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:40,892 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:40,895 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,895 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 11:53:40,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:40,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:40,895 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:40,895 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 11:53:40,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:40,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:40,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:40,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:40,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 11:53:40,898 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 11:53:40,899 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:40,899 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:40,899 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,899 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:40,901 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,901 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:40,901 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:40,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:40,902 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:40,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:40,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,905 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:40,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,914 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:40,914 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,914 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:40,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:40,914 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:40,914 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:40,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:40,915 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:40,915 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:40,915 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,915 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,915 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:40,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:40,917 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:40,917 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:41,017 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:41,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:41,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:41,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:41,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,035 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:41,035 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:41,036 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 11:53:41,036 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 11:53:41,036 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,036 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,038 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:41,038 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:41,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:41,038 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,038 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:41,039 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:41,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,039 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,039 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 11:53:41,040 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,040 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 11:53:41,040 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:41,040 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 11:53:41,041 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:41,041 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:41,041 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:41,041 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. started... 2023-03-08 11:53:41,042 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 09169a8d7fcfaa680f26277a4640bff1 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 11:53:41,469 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ca5748209798410da6b98a6377ca1455 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ca5748209798410da6b98a6377ca1455 2023-03-08 11:53:41,880 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/8ad2413999f74b4c8a28df2835b9bd19 2023-03-08 11:53:41,891 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/8ad2413999f74b4c8a28df2835b9bd19 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/8ad2413999f74b4c8a28df2835b9bd19 2023-03-08 11:53:41,899 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/8ad2413999f74b4c8a28df2835b9bd19, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 11:53:41,901 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 09169a8d7fcfaa680f26277a4640bff1 in 859ms, sequenceid=9, compaction requested=false 2023-03-08 11:53:41,901 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:41,901 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:41,901 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:41,901 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:41,901 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,901 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 11:53:41,901 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 11:53:41,904 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,904 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:41,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:41,905 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,905 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:41,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:41,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:41,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 11:53:41,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cf4da5d[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:41,907 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 11:53:41,907 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,908 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,908 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 11:53:41,908 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 11:53:41,908 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:41,908 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,909 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:41,910 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 11:53:41,910 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:41,910 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,910 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 11:53:41,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:41,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:41,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:41,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:41,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:41,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:41,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 11:53:41,914 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 11:53:41,914 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:41,914 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:41,914 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,914 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:41,919 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,919 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:41,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:41,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,919 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:41,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:41,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:41,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:41,920 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:41,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,922 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:41,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,925 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,925 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,925 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:41,925 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:41,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:41,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:41,925 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:41,925 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:41,925 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:41,926 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:41,926 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,926 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:41,926 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:41,927 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:41,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:42,027 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:42,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:42,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:42,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:42,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,040 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:42,040 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:42,041 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 11:53:42,041 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 11:53:42,041 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,041 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,043 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,043 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:42,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:42,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:42,043 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,043 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:42,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:42,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,044 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,044 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 11:53:42,044 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,045 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 11:53:42,045 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:42,045 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 11:53:42,045 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:42,045 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:42,045 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:42,045 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. started... 2023-03-08 11:53:42,045 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 09169a8d7fcfaa680f26277a4640bff1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:42,146 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:53:42,219 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-08 11:53:42,220 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-08 11:53:42,474 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201bed50b8ff2454d17ae6eace551c6d24f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201bed50b8ff2454d17ae6eace551c6d24f 2023-03-08 11:53:42,886 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/d584ba1a1f344e3aba9de05ecb444383 2023-03-08 11:53:42,895 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/d584ba1a1f344e3aba9de05ecb444383 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/d584ba1a1f344e3aba9de05ecb444383 2023-03-08 11:53:42,904 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/d584ba1a1f344e3aba9de05ecb444383, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 11:53:42,906 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 09169a8d7fcfaa680f26277a4640bff1 in 861ms, sequenceid=13, compaction requested=true 2023-03-08 11:53:42,906 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:42,906 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:42,906 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:42,906 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:42,906 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,906 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 11:53:42,907 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 11:53:42,909 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,909 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:42,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:42,909 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,910 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:42,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:42,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:42,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:42,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 11:53:42,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6578a8ae[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:42,912 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 11:53:42,912 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,914 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,914 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 11:53:42,914 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 11:53:42,914 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:42,914 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,914 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:42,916 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 11:53:42,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:42,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:42,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:42,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 11:53:42,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:42,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:42,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:42,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:42,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 11:53:42,920 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 11:53:42,920 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:42,920 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:42,920 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,920 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:42,930 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,930 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,930 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:42,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:42,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:42,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:42,931 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:42,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:42,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:42,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,933 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,938 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,938 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,938 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,938 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:42,938 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:42,938 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:42,938 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,938 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:42,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:42,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:42,938 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:42,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:42,939 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:42,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 11:53:42,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:42,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:42,940 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:42,940 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:43,040 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:43,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:43,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:43,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:43,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,055 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:43,055 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:43,056 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 11:53:43,056 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 11:53:43,056 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,057 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,058 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,058 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:43,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:43,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:43,059 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,059 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:43,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:43,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,061 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 11:53:43,061 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,061 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 11:53:43,061 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:43,062 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 11:53:43,062 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:43,062 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:43,062 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:43,062 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. started... 2023-03-08 11:53:43,063 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 09169a8d7fcfaa680f26277a4640bff1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:43,089 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e2469f209a444bc5841f3e223d87a0a4 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205e2469f209a444bc5841f3e223d87a0a4 2023-03-08 11:53:43,501 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/08320c6684e5444f89e582127c659c97 2023-03-08 11:53:43,510 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/08320c6684e5444f89e582127c659c97 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/08320c6684e5444f89e582127c659c97 2023-03-08 11:53:43,519 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/08320c6684e5444f89e582127c659c97, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 11:53:43,520 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 09169a8d7fcfaa680f26277a4640bff1 in 457ms, sequenceid=17, compaction requested=true 2023-03-08 11:53:43,520 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:43,520 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:43,520 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:43,521 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:43,521 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,521 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 11:53:43,521 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 11:53:43,523 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,523 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:43,524 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,524 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:43,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:43,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:43,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:43,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 11:53:43,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7003b17[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:43,526 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 11:53:43,526 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,528 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,528 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 11:53:43,528 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 11:53:43,528 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:43,528 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,528 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:43,530 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 11:53:43,530 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,530 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:43,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:43,530 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 11:53:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:43,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 11:53:43,534 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 11:53:43,535 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:43,535 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:43,535 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,535 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:43,536 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,536 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,536 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:43,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,538 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,539 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:43,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,544 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,544 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,544 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,544 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:43,544 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:43,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:43,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:43,545 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:43,545 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 11:53:43,545 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:43,545 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,546 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,546 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,546 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:43,546 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:43,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:43,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:43,646 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:43,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:43,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:43,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:43,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,658 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:43,658 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:43,661 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 11:53:43,661 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 11:53:43,662 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,662 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,664 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,664 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:43,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:43,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:43,664 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:43,665 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:43,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,665 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:43,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,666 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,666 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 11:53:43,666 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:43,666 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 11:53:43,666 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:43,666 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 11:53:43,667 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:43,667 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:43,667 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:43,667 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. started... 2023-03-08 11:53:43,667 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 09169a8d7fcfaa680f26277a4640bff1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:43,899 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-08 11:53:43,900 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-08 11:53:44,092 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512283011df32692c4dd19223757464212b51 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283011df32692c4dd19223757464212b51 2023-03-08 11:53:44,105 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/e78d5519cec6432989dfe7f5b57774a6 2023-03-08 11:53:44,113 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/e78d5519cec6432989dfe7f5b57774a6 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e78d5519cec6432989dfe7f5b57774a6 2023-03-08 11:53:44,121 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e78d5519cec6432989dfe7f5b57774a6, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 11:53:44,122 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 09169a8d7fcfaa680f26277a4640bff1 in 455ms, sequenceid=21, compaction requested=true 2023-03-08 11:53:44,123 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:44,123 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 11:53:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 11:53:44,128 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:44,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:44,129 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,129 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:44,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:44,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 11:53:44,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7736a80[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:44,131 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 11:53:44,131 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,132 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,132 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 11:53:44,132 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 11:53:44,132 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:44,132 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,132 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:44,134 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 11:53:44,134 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,134 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:44,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,134 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 11:53:44,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:44,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:44,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:44,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:44,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:44,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:44,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 11:53:44,137 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 11:53:44,137 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:44,137 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:44,137 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,137 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:44,138 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,138 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:44,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,138 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:44,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:44,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:44,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:44,139 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,141 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:44,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,149 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,149 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:44,149 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:44,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:44,149 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,149 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:44,149 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:44,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:44,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:44,149 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:44,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:44,149 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,149 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 11:53:44,150 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,150 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,150 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:44,150 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:44,251 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:44,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:44,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:44,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:44,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,262 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:44,262 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:44,263 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 11:53:44,263 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 11:53:44,263 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,263 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,265 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:44,265 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:44,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:44,265 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:44,265 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:44,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,266 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,266 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,266 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 11:53:44,267 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:44,267 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 11:53:44,267 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:44,267 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 11:53:44,267 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:44,267 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:44,267 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:44,267 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. started... 2023-03-08 11:53:44,268 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 09169a8d7fcfaa680f26277a4640bff1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:44,691 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512314ac2007777264d75bfe40b13951757fb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512314ac2007777264d75bfe40b13951757fb 2023-03-08 11:53:45,103 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/b059b54f1f174bfbb926157012bf564c 2023-03-08 11:53:45,112 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/b059b54f1f174bfbb926157012bf564c as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/b059b54f1f174bfbb926157012bf564c 2023-03-08 11:53:45,119 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/b059b54f1f174bfbb926157012bf564c, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 11:53:45,121 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 09169a8d7fcfaa680f26277a4640bff1 in 853ms, sequenceid=25, compaction requested=true 2023-03-08 11:53:45,121 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:45,121 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:45,121 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:45,121 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:45,122 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,122 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 11:53:45,122 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 11:53:45,123 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,123 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:45,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:45,124 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,124 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:45,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:45,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:45,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 11:53:45,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2dff6e01[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:45,126 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 11:53:45,126 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,127 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 11:53:45,128 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 11:53:45,128 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:45,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:45,135 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 11:53:45,135 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,135 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:45,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:45,135 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 11:53:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:45,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:45,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 11:53:45,139 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 11:53:45,139 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:45,139 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:45,139 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,139 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:45,141 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,141 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,141 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:45,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:45,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:45,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:45,141 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:45,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:45,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:45,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,143 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:45,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,173 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,173 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,174 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:45,174 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:45,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:45,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:45,175 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,175 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,175 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:45,175 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 11:53:45,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:45,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:45,175 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,176 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,176 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:45,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:45,176 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:45,176 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:45,276 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:45,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:45,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:45,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:45,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,288 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:45,288 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:45,288 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 11:53:45,288 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 11:53:45,289 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,289 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,290 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,290 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:45,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:45,291 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:45,291 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:45,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,292 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:45,293 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 11:53:45,293 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:45,293 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 11:53:45,293 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:45,293 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:45,293 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:45,293 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. started... 2023-03-08 11:53:45,294 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 09169a8d7fcfaa680f26277a4640bff1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:45,714 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101b87d559cbb6144c885f6454c141fc6fe to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101b87d559cbb6144c885f6454c141fc6fe 2023-03-08 11:53:46,153 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/375e3031b0ad4c799429146e84561084 2023-03-08 11:53:46,162 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/375e3031b0ad4c799429146e84561084 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/375e3031b0ad4c799429146e84561084 2023-03-08 11:53:46,172 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/375e3031b0ad4c799429146e84561084, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 11:53:46,173 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 09169a8d7fcfaa680f26277a4640bff1 in 879ms, sequenceid=29, compaction requested=true 2023-03-08 11:53:46,174 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:46,174 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:46,174 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 11:53:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 11:53:46,178 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,178 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:46,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:46,178 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,178 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:46,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:46,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:46,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:46,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 11:53:46,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d628ec8[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:46,180 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 11:53:46,180 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,181 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,182 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 11:53:46,182 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 11:53:46,182 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:46,182 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,182 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:46,183 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,183 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 11:53:46,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:46,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:46,183 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:46,183 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 11:53:46,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:46,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:46,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:46,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 11:53:46,186 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 11:53:46,187 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:46,187 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:46,187 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,187 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:46,189 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,189 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,189 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:46,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:46,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:46,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:46,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:46,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:46,190 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:46,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,191 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:46,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,199 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,199 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:46,199 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:46,199 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:46,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:46,199 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:46,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:46,199 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:46,199 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,200 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 11:53:46,200 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,200 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,200 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:46,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:46,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:46,200 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,200 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:46,300 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:46,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:46,312 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:46,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:46,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,318 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:46,318 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:46,319 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 11:53:46,319 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 11:53:46,320 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,320 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,321 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,321 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:46,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:46,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:46,321 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,321 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:46,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:46,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,323 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,324 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 11:53:46,324 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,324 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 11:53:46,324 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:46,325 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 11:53:46,325 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:46,325 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:46,325 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:46,325 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. started... 2023-03-08 11:53:46,325 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 09169a8d7fcfaa680f26277a4640bff1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:46,357 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010345191f2b44914d599e1eecbb31fb7d89 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010345191f2b44914d599e1eecbb31fb7d89 2023-03-08 11:53:46,377 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/247c1c56d62b4579a95af87eb793dc02 2023-03-08 11:53:46,386 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/247c1c56d62b4579a95af87eb793dc02 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/247c1c56d62b4579a95af87eb793dc02 2023-03-08 11:53:46,395 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/247c1c56d62b4579a95af87eb793dc02, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 11:53:46,397 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 09169a8d7fcfaa680f26277a4640bff1 in 72ms, sequenceid=33, compaction requested=true 2023-03-08 11:53:46,397 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:46,397 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:46,397 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:46,397 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:46,397 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,397 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 11:53:46,398 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 11:53:46,399 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,399 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,400 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,400 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:46,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 11:53:46,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@bbcc4f0[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:46,402 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 11:53:46,402 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,403 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,404 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 11:53:46,404 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 11:53:46,404 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:46,404 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,404 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:46,410 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,410 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 11:53:46,410 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:46,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:46,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:46,410 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 11:53:46,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:46,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:46,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:46,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:46,415 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 11:53:46,415 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 11:53:46,415 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:46,415 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:46,415 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,415 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:46,417 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,417 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:46,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:46,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,417 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:46,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:46,418 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:46,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:46,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:46,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,420 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:46,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,423 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,423 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,423 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:46,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:46,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:46,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:46,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 11:53:46,423 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:46,423 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:46,423 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:46,424 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,424 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,424 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:46,425 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:46,424 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,425 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:46,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:46,525 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:46,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:46,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:46,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:46,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,541 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:46,541 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:46,541 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 11:53:46,542 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 11:53:46,542 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,542 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,545 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:46,545 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:46,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:46,545 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:46,545 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:46,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:46,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,546 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,546 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 11:53:46,546 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:46,546 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 11:53:46,546 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:46,547 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 11:53:46,547 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:46,547 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:46,547 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:46,547 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. started... 2023-03-08 11:53:46,547 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 09169a8d7fcfaa680f26277a4640bff1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:46,974 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907aa9f0416df974a909c834b5646d71a4e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20150907aa9f0416df974a909c834b5646d71a4e 2023-03-08 11:53:46,989 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/e04ef2648215492190fcc0e0cf31ebe3 2023-03-08 11:53:47,001 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/e04ef2648215492190fcc0e0cf31ebe3 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e04ef2648215492190fcc0e0cf31ebe3 2023-03-08 11:53:47,009 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e04ef2648215492190fcc0e0cf31ebe3, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 11:53:47,012 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 09169a8d7fcfaa680f26277a4640bff1 in 465ms, sequenceid=37, compaction requested=true 2023-03-08 11:53:47,012 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:47,012 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:47,012 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 11:53:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 11:53:47,016 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,016 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:47,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:47,016 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,016 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:47,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:47,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:47,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:47,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 11:53:47,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1787986d[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:47,018 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 11:53:47,018 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,019 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,020 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 11:53:47,020 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 11:53:47,020 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:47,020 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,020 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:47,021 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 11:53:47,021 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,021 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:47,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:47,022 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 11:53:47,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:47,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:47,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:47,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 11:53:47,026 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 11:53:47,026 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:47,026 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:47,026 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,026 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:47,029 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,029 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:47,029 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,030 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:47,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:47,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:47,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:47,031 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,034 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,034 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,034 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,034 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,034 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:47,034 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:47,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:47,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:47,034 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:47,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:47,034 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 11:53:47,034 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:47,035 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,035 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:47,035 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:47,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:47,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:47,136 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:47,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:47,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:47,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:47,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,146 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:47,146 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:47,147 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 11:53:47,147 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 11:53:47,147 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,148 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,149 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,149 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:47,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:47,149 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,149 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,150 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,150 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 11:53:47,151 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:47,151 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 11:53:47,151 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:47,152 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 11:53:47,152 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:47,152 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:47,152 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:47,152 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. started... 2023-03-08 11:53:47,152 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 09169a8d7fcfaa680f26277a4640bff1 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:47,573 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120cae15df4d30e422584061fed80b3d7a6 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120cae15df4d30e422584061fed80b3d7a6 2023-03-08 11:53:47,982 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/467c1298ce344085a656c8f1e8d30276 2023-03-08 11:53:47,990 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/467c1298ce344085a656c8f1e8d30276 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/467c1298ce344085a656c8f1e8d30276 2023-03-08 11:53:47,997 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/467c1298ce344085a656c8f1e8d30276, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 11:53:47,998 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 09169a8d7fcfaa680f26277a4640bff1 in 846ms, sequenceid=41, compaction requested=true 2023-03-08 11:53:47,998 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:47,998 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:47,998 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:47,999 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:47,999 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:47,999 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 11:53:47,999 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 11:53:48,001 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,001 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:48,001 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,001 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:48,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:48,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 11:53:48,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41de63fa[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:48,003 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 11:53:48,003 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,004 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 11:53:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 11:53:48,004 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,004 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:48,010 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 11:53:48,010 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,010 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:48,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:48,010 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 11:53:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:48,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:48,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 11:53:48,015 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 11:53:48,015 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:48,015 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:48,015 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,015 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:48,017 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,017 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:48,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,017 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:48,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:48,018 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,019 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,021 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,021 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:48,021 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:48,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:48,022 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 11:53:48,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:48,022 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:48,022 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:48,022 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:48,022 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:48,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:48,022 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,022 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:48,022 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:48,022 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,123 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:48,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:48,129 DEBUG [Listener at localhost/36211] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 11:53:48,131 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58102, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 11:53:48,133 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:48,145 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 11:53:48,146 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 11:53:48,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:53:48,147 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277028147 2023-03-08 11:53:48,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:53:48,153 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-08 11:53:48,153 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 11:53:48,153 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x4e76a679 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:53:48,157 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17de54c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:53:48,159 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:49,032 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037a315eed1bba461fa7a3c6635a3b6747 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601037a315eed1bba461fa7a3c6635a3b6747 2023-03-08 11:53:49,033 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 11:53:49,053 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:53:49,055 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38078, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:53:49,066 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:53:49,068 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58104, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:53:49,102 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b0a1b572f9074a1ab758282d589bf81f first=Optional[r3] last=Optional[r6] 2023-03-08 11:53:49,116 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b0a1b572f9074a1ab758282d589bf81f}] 2023-03-08 11:53:49,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b0a1b572f9074a1ab758282d589bf81f for inclusion in 09169a8d7fcfaa680f26277a4640bff1/family1 2023-03-08 11:53:49,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 11:53:49,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:53:49,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:49,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b0a1b572f9074a1ab758282d589bf81f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__baf90ff7qja7nvtukkc2hdujnqje6sonve3d7qjqnq03vqc2v6pfao9kehs4npi2/family1/b0a1b572f9074a1ab758282d589bf81f 2023-03-08 11:53:49,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__baf90ff7qja7nvtukkc2hdujnqje6sonve3d7qjqnq03vqc2v6pfao9kehs4npi2/family1/b0a1b572f9074a1ab758282d589bf81f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ce0b24b3f35a454d823b6f4d855db6f4_SeqId_44_ 2023-03-08 11:53:49,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__baf90ff7qja7nvtukkc2hdujnqje6sonve3d7qjqnq03vqc2v6pfao9kehs4npi2/family1/b0a1b572f9074a1ab758282d589bf81f into 09169a8d7fcfaa680f26277a4640bff1/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ce0b24b3f35a454d823b6f4d855db6f4_SeqId_44_ - updating store file list. 2023-03-08 11:53:49,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ce0b24b3f35a454d823b6f4d855db6f4_SeqId_44_ into 09169a8d7fcfaa680f26277a4640bff1/family1 2023-03-08 11:53:49,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__baf90ff7qja7nvtukkc2hdujnqje6sonve3d7qjqnq03vqc2v6pfao9kehs4npi2/family1/b0a1b572f9074a1ab758282d589bf81f into 09169a8d7fcfaa680f26277a4640bff1/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ce0b24b3f35a454d823b6f4d855db6f4_SeqId_44_) 2023-03-08 11:53:49,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__baf90ff7qja7nvtukkc2hdujnqje6sonve3d7qjqnq03vqc2v6pfao9kehs4npi2/family1/b0a1b572f9074a1ab758282d589bf81f 2023-03-08 11:53:49,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 11:53:49,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 11:53:49,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 09169a8d7fcfaa680f26277a4640bff1/family1 is initiating minor compaction 2023-03-08 11:53:49,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@661ea7bf; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:53:49,170 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 09169a8d7fcfaa680f26277a4640bff1/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:49,171 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/8ad2413999f74b4c8a28df2835b9bd19, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/d584ba1a1f344e3aba9de05ecb444383, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/08320c6684e5444f89e582127c659c97, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e78d5519cec6432989dfe7f5b57774a6, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/b059b54f1f174bfbb926157012bf564c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/375e3031b0ad4c799429146e84561084, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/247c1c56d62b4579a95af87eb793dc02, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e04ef2648215492190fcc0e0cf31ebe3, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/467c1298ce344085a656c8f1e8d30276, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ce0b24b3f35a454d823b6f4d855db6f4_SeqId_44_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp, totalSize=50.2 K 2023-03-08 11:53:49,172 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 8ad2413999f74b4c8a28df2835b9bd19, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 11:53:49,172 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting d584ba1a1f344e3aba9de05ecb444383, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 11:53:49,172 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 08320c6684e5444f89e582127c659c97, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 11:53:49,172 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting e78d5519cec6432989dfe7f5b57774a6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 11:53:49,172 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting b059b54f1f174bfbb926157012bf564c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 11:53:49,172 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 375e3031b0ad4c799429146e84561084, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 11:53:49,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 247c1c56d62b4579a95af87eb793dc02, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 11:53:49,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting e04ef2648215492190fcc0e0cf31ebe3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 11:53:49,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 467c1298ce344085a656c8f1e8d30276, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 11:53:49,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting ce0b24b3f35a454d823b6f4d855db6f4_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 11:53:49,188 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 11:53:49,190 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:53:49,196 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283011df32692c4dd19223757464212b51 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283011df32692c4dd19223757464212b51 2023-03-08 11:53:49,198 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512314ac2007777264d75bfe40b13951757fb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512314ac2007777264d75bfe40b13951757fb 2023-03-08 11:53:49,200 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101b87d559cbb6144c885f6454c141fc6fe to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101b87d559cbb6144c885f6454c141fc6fe 2023-03-08 11:53:49,201 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010345191f2b44914d599e1eecbb31fb7d89 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010345191f2b44914d599e1eecbb31fb7d89 2023-03-08 11:53:49,201 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-08 11:53:49,201 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 11:53:49,202 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:49,202 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:49,202 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:49,202 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:49,202 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:49,202 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:49,202 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:49,202 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:49,231 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 09169a8d7fcfaa680f26277a4640bff1#family1#compaction#10 average throughput is 0.48 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:53:49,666 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/bae1cad78f9f45aaacc088a088a67ab8 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/bae1cad78f9f45aaacc088a088a67ab8 2023-03-08 11:53:49,682 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 09169a8d7fcfaa680f26277a4640bff1/family1 of 09169a8d7fcfaa680f26277a4640bff1 into bae1cad78f9f45aaacc088a088a67ab8(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:53:49,682 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:49,684 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1678276429168; duration=0sec 2023-03-08 11:53:49,684 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:53:49,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/8ad2413999f74b4c8a28df2835b9bd19, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/d584ba1a1f344e3aba9de05ecb444383, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/08320c6684e5444f89e582127c659c97, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e78d5519cec6432989dfe7f5b57774a6, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/b059b54f1f174bfbb926157012bf564c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/375e3031b0ad4c799429146e84561084, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/247c1c56d62b4579a95af87eb793dc02, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e04ef2648215492190fcc0e0cf31ebe3, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/467c1298ce344085a656c8f1e8d30276, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ce0b24b3f35a454d823b6f4d855db6f4_SeqId_44_] to archive 2023-03-08 11:53:49,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:53:49,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/8ad2413999f74b4c8a28df2835b9bd19 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/8ad2413999f74b4c8a28df2835b9bd19 2023-03-08 11:53:49,782 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/d584ba1a1f344e3aba9de05ecb444383 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/d584ba1a1f344e3aba9de05ecb444383 2023-03-08 11:53:49,784 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/08320c6684e5444f89e582127c659c97 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/08320c6684e5444f89e582127c659c97 2023-03-08 11:53:49,790 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e78d5519cec6432989dfe7f5b57774a6 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e78d5519cec6432989dfe7f5b57774a6 2023-03-08 11:53:49,792 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/b059b54f1f174bfbb926157012bf564c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/b059b54f1f174bfbb926157012bf564c 2023-03-08 11:53:49,794 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/375e3031b0ad4c799429146e84561084 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/375e3031b0ad4c799429146e84561084 2023-03-08 11:53:49,795 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/247c1c56d62b4579a95af87eb793dc02 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/247c1c56d62b4579a95af87eb793dc02 2023-03-08 11:53:49,797 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e04ef2648215492190fcc0e0cf31ebe3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e04ef2648215492190fcc0e0cf31ebe3 2023-03-08 11:53:49,798 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/467c1298ce344085a656c8f1e8d30276 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/467c1298ce344085a656c8f1e8d30276 2023-03-08 11:53:49,800 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ce0b24b3f35a454d823b6f4d855db6f4_SeqId_44_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ce0b24b3f35a454d823b6f4d855db6f4_SeqId_44_ 2023-03-08 11:53:50,103 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057f33cd87a62842fa9e3c8b03c72e1648 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512057f33cd87a62842fa9e3c8b03c72e1648 2023-03-08 11:53:50,105 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 11:53:50,136 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/eec07082f23542319fb6fc906150fc37 first=Optional[r00] last=Optional[r2] 2023-03-08 11:53:50,137 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/eec07082f23542319fb6fc906150fc37}] 2023-03-08 11:53:50,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/eec07082f23542319fb6fc906150fc37 for inclusion in 09169a8d7fcfaa680f26277a4640bff1/family1 2023-03-08 11:53:50,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 11:53:50,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:53:50,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:50,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/eec07082f23542319fb6fc906150fc37 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kf8d993q167ot2p0c0tthr1a2jglalkkepmrdqa5saigagabqba11f753bq2pd4k/family1/eec07082f23542319fb6fc906150fc37 2023-03-08 11:53:50,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kf8d993q167ot2p0c0tthr1a2jglalkkepmrdqa5saigagabqba11f753bq2pd4k/family1/eec07082f23542319fb6fc906150fc37 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/f451e24bf54749df85d34a0a5c4b0d58_SeqId_47_ 2023-03-08 11:53:50,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kf8d993q167ot2p0c0tthr1a2jglalkkepmrdqa5saigagabqba11f753bq2pd4k/family1/eec07082f23542319fb6fc906150fc37 into 09169a8d7fcfaa680f26277a4640bff1/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/f451e24bf54749df85d34a0a5c4b0d58_SeqId_47_ - updating store file list. 2023-03-08 11:53:50,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/f451e24bf54749df85d34a0a5c4b0d58_SeqId_47_ into 09169a8d7fcfaa680f26277a4640bff1/family1 2023-03-08 11:53:50,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kf8d993q167ot2p0c0tthr1a2jglalkkepmrdqa5saigagabqba11f753bq2pd4k/family1/eec07082f23542319fb6fc906150fc37 into 09169a8d7fcfaa680f26277a4640bff1/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/f451e24bf54749df85d34a0a5c4b0d58_SeqId_47_) 2023-03-08 11:53:50,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kf8d993q167ot2p0c0tthr1a2jglalkkepmrdqa5saigagabqba11f753bq2pd4k/family1/eec07082f23542319fb6fc906150fc37 2023-03-08 11:53:50,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 11:53:50,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 11:53:50,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 09169a8d7fcfaa680f26277a4640bff1/family1 is initiating minor compaction (all files) 2023-03-08 11:53:50,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@77be3773; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:53:50,159 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 09169a8d7fcfaa680f26277a4640bff1/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:50,160 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e355568068724fa787b45dbf02f4b995, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/bae1cad78f9f45aaacc088a088a67ab8, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/f451e24bf54749df85d34a0a5c4b0d58_SeqId_47_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp, totalSize=358.6 K 2023-03-08 11:53:50,160 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting e355568068724fa787b45dbf02f4b995, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838541 2023-03-08 11:53:50,161 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting bae1cad78f9f45aaacc088a088a67ab8, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238541 2023-03-08 11:53:50,161 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting f451e24bf54749df85d34a0a5c4b0d58_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878838541 2023-03-08 11:53:50,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 11:53:50,163 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:53:50,165 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113034155f03a9014456b44d4079a48c520f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113034155f03a9014456b44d4079a48c520f 2023-03-08 11:53:50,167 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201bed50b8ff2454d17ae6eace551c6d24f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201bed50b8ff2454d17ae6eace551c6d24f 2023-03-08 11:53:50,170 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205e2469f209a444bc5841f3e223d87a0a4 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205e2469f209a444bc5841f3e223d87a0a4 2023-03-08 11:53:50,170 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-08 11:53:50,170 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 11:53:50,171 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:50,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 11:53:50,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:53:50,173 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 11:53:50,175 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 2.0350 sec 2023-03-08 11:53:50,205 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 09169a8d7fcfaa680f26277a4640bff1#family1#compaction#11 average throughput is 5.90 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:53:50,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 11:53:50,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:50,339 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-08 11:53:50,340 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-08 11:53:50,340 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-08 11:53:50,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-08 11:53:50,357 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=09169a8d7fcfaa680f26277a4640bff1, REOPEN/MOVE}] 2023-03-08 11:53:50,358 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=09169a8d7fcfaa680f26277a4640bff1, REOPEN/MOVE 2023-03-08 11:53:50,359 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=09169a8d7fcfaa680f26277a4640bff1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:50,359 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678276430359"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276430359"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276430359"}]},"ts":"1678276430359"} 2023-03-08 11:53:50,362 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 09169a8d7fcfaa680f26277a4640bff1, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:53:50,521 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:50,521 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 09169a8d7fcfaa680f26277a4640bff1, disabling compactions & flushes 2023-03-08 11:53:50,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:50,633 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/.tmp/family1/c0c03f5746b04cf79b3d8bc6cb777065 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/c0c03f5746b04cf79b3d8bc6cb777065 2023-03-08 11:53:50,640 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 09169a8d7fcfaa680f26277a4640bff1/family1 of 09169a8d7fcfaa680f26277a4640bff1 into c0c03f5746b04cf79b3d8bc6cb777065(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:53:50,641 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:50,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:50,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:50,641 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1678276430159; duration=0sec 2023-03-08 11:53:50,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. after waiting 0 ms 2023-03-08 11:53:50,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:50,641 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:53:50,645 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e355568068724fa787b45dbf02f4b995, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/bae1cad78f9f45aaacc088a088a67ab8, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/f451e24bf54749df85d34a0a5c4b0d58_SeqId_47_] to archive 2023-03-08 11:53:50,646 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:53:50,648 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e355568068724fa787b45dbf02f4b995 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/e355568068724fa787b45dbf02f4b995 2023-03-08 11:53:50,649 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/bae1cad78f9f45aaacc088a088a67ab8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/bae1cad78f9f45aaacc088a088a67ab8 2023-03-08 11:53:50,651 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/f451e24bf54749df85d34a0a5c4b0d58_SeqId_47_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/f451e24bf54749df85d34a0a5c4b0d58_SeqId_47_ 2023-03-08 11:53:50,656 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 11:53:50,658 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:50,658 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:50,659 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 09169a8d7fcfaa680f26277a4640bff1 to self. 2023-03-08 11:53:50,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:50,661 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=09169a8d7fcfaa680f26277a4640bff1, regionState=CLOSED 2023-03-08 11:53:50,662 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678276430661"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276430661"}]},"ts":"1678276430661"} 2023-03-08 11:53:50,665 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-08 11:53:50,666 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 09169a8d7fcfaa680f26277a4640bff1, server=jenkins-hbase4.apache.org,45551,1678276414946 in 301 msec 2023-03-08 11:53:50,666 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=09169a8d7fcfaa680f26277a4640bff1, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=true 2023-03-08 11:53:50,817 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=09169a8d7fcfaa680f26277a4640bff1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:50,817 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678276430817"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276430817"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276430817"}]},"ts":"1678276430817"} 2023-03-08 11:53:50,819 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 09169a8d7fcfaa680f26277a4640bff1, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:53:50,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:50,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 09169a8d7fcfaa680f26277a4640bff1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:53:50,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:50,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:50,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:50,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:50,978 INFO [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:50,979 DEBUG [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1 2023-03-08 11:53:50,979 DEBUG [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1 2023-03-08 11:53:50,979 INFO [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09169a8d7fcfaa680f26277a4640bff1 columnFamilyName family1 2023-03-08 11:53:50,987 DEBUG [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/c0c03f5746b04cf79b3d8bc6cb777065 2023-03-08 11:53:50,987 INFO [StoreOpener-09169a8d7fcfaa680f26277a4640bff1-1] regionserver.HStore(310): Store=09169a8d7fcfaa680f26277a4640bff1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:50,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:50,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:50,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:53:50,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 09169a8d7fcfaa680f26277a4640bff1; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70710476, jitterRate=0.053668200969696045}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:53:50,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:50,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1., pid=17, masterSystemTime=1678276430972 2023-03-08 11:53:50,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:50,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:53:50,997 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=09169a8d7fcfaa680f26277a4640bff1, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:50,997 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678276430997"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276430997"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276430997"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276430997"}]},"ts":"1678276430997"} 2023-03-08 11:53:51,001 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-08 11:53:51,001 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 09169a8d7fcfaa680f26277a4640bff1, server=jenkins-hbase4.apache.org,45551,1678276414946 in 180 msec 2023-03-08 11:53:51,003 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-08 11:53:51,003 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=09169a8d7fcfaa680f26277a4640bff1, REOPEN/MOVE in 644 msec 2023-03-08 11:53:51,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-08 11:53:51,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 660 msec 2023-03-08 11:53:51,009 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 690 msec 2023-03-08 11:53:51,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-08 11:53:51,019 INFO [Listener at localhost/36211] compactions.TestMobCompactor(278): alter status finished 2023-03-08 11:53:51,020 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 11:53:51,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 11:53:51,023 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 11:53:51,024 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277031024 2023-03-08 11:53:51,024 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:53:51,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:53:51,026 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-08 11:53:51,026 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 11:53:51,026 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x127ee2f7 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:53:51,032 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3bc991e5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:53:51,033 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:51,033 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:51,033 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:51,033 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:51,033 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:51,867 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280e9e8bf2d4b145278ff6fe92f807801e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511280e9e8bf2d4b145278ff6fe92f807801e 2023-03-08 11:53:51,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 11:53:51,880 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:53:51,882 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38080, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:53:51,889 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:53:51,891 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58118, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:53:51,902 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a91d2a962588451cb0948c2862a1f8c2 first=Optional[r05] last=Optional[r8] 2023-03-08 11:53:51,903 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a91d2a962588451cb0948c2862a1f8c2}] 2023-03-08 11:53:51,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a91d2a962588451cb0948c2862a1f8c2 for inclusion in 09169a8d7fcfaa680f26277a4640bff1/family1 2023-03-08 11:53:51,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 11:53:51,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:53:51,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:53:51,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a91d2a962588451cb0948c2862a1f8c2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__dra32hm402j5i7s7e107ntavgf31tqg472k75evg0ncs02bm9bankach0smdrkm6/family1/a91d2a962588451cb0948c2862a1f8c2 2023-03-08 11:53:51,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__dra32hm402j5i7s7e107ntavgf31tqg472k75evg0ncs02bm9bankach0smdrkm6/family1/a91d2a962588451cb0948c2862a1f8c2 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ab25fefb740e49c6b5d4b262d4543156_SeqId_53_ 2023-03-08 11:53:51,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__dra32hm402j5i7s7e107ntavgf31tqg472k75evg0ncs02bm9bankach0smdrkm6/family1/a91d2a962588451cb0948c2862a1f8c2 into 09169a8d7fcfaa680f26277a4640bff1/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ab25fefb740e49c6b5d4b262d4543156_SeqId_53_ - updating store file list. 2023-03-08 11:53:51,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ab25fefb740e49c6b5d4b262d4543156_SeqId_53_ into 09169a8d7fcfaa680f26277a4640bff1/family1 2023-03-08 11:53:51,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__dra32hm402j5i7s7e107ntavgf31tqg472k75evg0ncs02bm9bankach0smdrkm6/family1/a91d2a962588451cb0948c2862a1f8c2 into 09169a8d7fcfaa680f26277a4640bff1/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/family1/ab25fefb740e49c6b5d4b262d4543156_SeqId_53_) 2023-03-08 11:53:51,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__dra32hm402j5i7s7e107ntavgf31tqg472k75evg0ncs02bm9bankach0smdrkm6/family1/a91d2a962588451cb0948c2862a1f8c2 2023-03-08 11:53:51,924 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 11:53:51,925 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:53:51,926 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120cae15df4d30e422584061fed80b3d7a6 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120cae15df4d30e422584061fed80b3d7a6 2023-03-08 11:53:51,927 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ca5748209798410da6b98a6377ca1455 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ca5748209798410da6b98a6377ca1455 2023-03-08 11:53:51,928 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-08 11:53:51,928 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 11:53:51,928 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:51,928 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:51,928 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 11:53:51,928 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 11:53:51,928 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:53:51,929 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 11:53:51,931 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 908 msec 2023-03-08 11:53:52,012 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=250 (was 209) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36232 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36278 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/cluster_7d11bbd1-2ad2-65f1-e179-3d37987b50d1/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36212 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36188 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:45551-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36414 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x34aa4df9-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36380 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36398 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51863@0x4e76a679 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1824476703.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x34aa4df9-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51863@0x127ee2f7-SendThread(127.0.0.1:51863) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x34aa4df9-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51863@0x127ee2f7-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51863@0x127ee2f7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1824476703.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36324 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36302 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36312 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e36b3a5-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x34aa4df9-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36364 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36422 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51863@0x4e76a679-SendThread(127.0.0.1:51863) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x7e36b3a5-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36250 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36412 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e36b3a5-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_735287611_15 at /127.0.0.1:36292 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51863@0x4e76a679-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x34aa4df9-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:36472 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=593 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=234 (was 292), ProcessCount=170 (was 170), AvailableMemoryMB=7455 (was 7721) 2023-03-08 11:53:52,024 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=250, OpenFileDescriptor=593, MaxFileDescriptor=60000, SystemLoadAverage=234, ProcessCount=170, AvailableMemoryMB=7455 2023-03-08 11:53:52,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 11:53:52,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:52,029 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:53:52,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-08 11:53:52,030 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:53:52,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 11:53:52,031 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918 2023-03-08 11:53:52,032 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918 empty. 2023-03-08 11:53:52,033 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918 2023-03-08 11:53:52,033 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-08 11:53:52,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 11:53:52,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 11:53:52,442 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 11:53:52,444 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 148f994b12b7c170e5fbb2fd36359918, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:53:52,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 11:53:52,851 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:52,851 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 148f994b12b7c170e5fbb2fd36359918, disabling compactions & flushes 2023-03-08 11:53:52,851 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:52,851 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:52,851 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. after waiting 0 ms 2023-03-08 11:53:52,851 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:52,852 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:52,852 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:53:52,855 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:53:52,856 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678276432855"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276432855"}]},"ts":"1678276432855"} 2023-03-08 11:53:52,858 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 11:53:52,859 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:53:52,859 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276432859"}]},"ts":"1678276432859"} 2023-03-08 11:53:52,861 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-08 11:53:52,864 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=148f994b12b7c170e5fbb2fd36359918, ASSIGN}] 2023-03-08 11:53:52,866 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=148f994b12b7c170e5fbb2fd36359918, ASSIGN 2023-03-08 11:53:52,867 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=148f994b12b7c170e5fbb2fd36359918, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:53:53,017 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=148f994b12b7c170e5fbb2fd36359918, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:53,018 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678276433017"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276433017"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276433017"}]},"ts":"1678276433017"} 2023-03-08 11:53:53,020 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 148f994b12b7c170e5fbb2fd36359918, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:53:53,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 11:53:53,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:53,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 148f994b12b7c170e5fbb2fd36359918, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:53:53,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 148f994b12b7c170e5fbb2fd36359918 2023-03-08 11:53:53,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:53:53,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 148f994b12b7c170e5fbb2fd36359918 2023-03-08 11:53:53,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 148f994b12b7c170e5fbb2fd36359918 2023-03-08 11:53:53,180 INFO [StoreOpener-148f994b12b7c170e5fbb2fd36359918-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 148f994b12b7c170e5fbb2fd36359918 2023-03-08 11:53:53,182 DEBUG [StoreOpener-148f994b12b7c170e5fbb2fd36359918-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1 2023-03-08 11:53:53,182 DEBUG [StoreOpener-148f994b12b7c170e5fbb2fd36359918-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1 2023-03-08 11:53:53,182 INFO [StoreOpener-148f994b12b7c170e5fbb2fd36359918-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 148f994b12b7c170e5fbb2fd36359918 columnFamilyName family1 2023-03-08 11:53:53,184 INFO [StoreOpener-148f994b12b7c170e5fbb2fd36359918-1] regionserver.HStore(310): Store=148f994b12b7c170e5fbb2fd36359918/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:53:53,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918 2023-03-08 11:53:53,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918 2023-03-08 11:53:53,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 148f994b12b7c170e5fbb2fd36359918 2023-03-08 11:53:53,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:53:53,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 148f994b12b7c170e5fbb2fd36359918; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61216852, jitterRate=-0.08779782056808472}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:53:53,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:53:53,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918., pid=21, masterSystemTime=1678276433172 2023-03-08 11:53:53,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:53,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:53,202 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=148f994b12b7c170e5fbb2fd36359918, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:53,202 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678276433202"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276433202"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276433202"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276433202"}]},"ts":"1678276433202"} 2023-03-08 11:53:53,210 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-08 11:53:53,210 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 148f994b12b7c170e5fbb2fd36359918, server=jenkins-hbase4.apache.org,45551,1678276414946 in 185 msec 2023-03-08 11:53:53,213 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-08 11:53:53,213 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=148f994b12b7c170e5fbb2fd36359918, ASSIGN in 346 msec 2023-03-08 11:53:53,213 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:53:53,214 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276433214"}]},"ts":"1678276433214"} 2023-03-08 11:53:53,215 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-08 11:53:53,218 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:53:53,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1920 sec 2023-03-08 11:53:54,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 11:53:54,136 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-08 11:53:54,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:54,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:54,169 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:54,169 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:54,170 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:53:54,170 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:53:54,170 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:54,170 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:54,172 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:54,172 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:54,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:54,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:54,173 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:54,173 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:54,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:54,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:54,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:54,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:54,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:54,174 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:54,174 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:53:54,174 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:54,175 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:53:54,175 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:54,175 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:54,175 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:54,175 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. started... 2023-03-08 11:53:54,176 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 148f994b12b7c170e5fbb2fd36359918 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 11:53:54,222 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:54,602 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113091abce7b0aa547c08961bf51004bbe45 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113091abce7b0aa547c08961bf51004bbe45 2023-03-08 11:53:55,006 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/358914a0885a4a3d88cb093aa0c91025 2023-03-08 11:53:55,014 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/358914a0885a4a3d88cb093aa0c91025 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/358914a0885a4a3d88cb093aa0c91025 2023-03-08 11:53:55,020 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/358914a0885a4a3d88cb093aa0c91025, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-08 11:53:55,021 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 148f994b12b7c170e5fbb2fd36359918 in 845ms, sequenceid=5, compaction requested=false 2023-03-08 11:53:55,022 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:53:55,022 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:55,022 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:55,022 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:55,022 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,022 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:53:55,022 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:53:55,023 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,023 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:55,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:55,024 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,024 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:55,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:55,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:55,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:55,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:53:55,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5853b1c7[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:55,025 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:53:55,025 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,027 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,027 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:53:55,027 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:53:55,027 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:55,027 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,027 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:55,030 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,030 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:53:55,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:55,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:55,030 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:55,030 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:53:55,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:55,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:55,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:55,033 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:53:55,033 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:53:55,033 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:55,033 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:55,033 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,033 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:55,035 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,035 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:55,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:55,035 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:55,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:55,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:55,035 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:55,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:55,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,037 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:55,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,039 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,039 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:55,039 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:55,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:55,039 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:55,039 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,039 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:55,039 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,039 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:55,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:55,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:55,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:55,040 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,040 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,040 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,040 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:55,040 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:55,141 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:55,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:55,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:55,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:55,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,150 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:55,150 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:55,150 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:53:55,150 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:53:55,151 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,151 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,152 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:55,152 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:55,153 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,153 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,153 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,154 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,154 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:53:55,154 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,154 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:53:55,154 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:55,155 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:53:55,157 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:55,157 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:55,157 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:55,157 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. started... 2023-03-08 11:53:55,157 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 148f994b12b7c170e5fbb2fd36359918 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 11:53:55,573 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128eeeaa333ff214e029853f6e0b762138e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128eeeaa333ff214e029853f6e0b762138e 2023-03-08 11:53:55,979 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/aa1ec47c53dd487282bf194c9e54f245 2023-03-08 11:53:55,986 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/aa1ec47c53dd487282bf194c9e54f245 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/aa1ec47c53dd487282bf194c9e54f245 2023-03-08 11:53:55,991 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/aa1ec47c53dd487282bf194c9e54f245, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 11:53:55,992 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 148f994b12b7c170e5fbb2fd36359918 in 835ms, sequenceid=9, compaction requested=false 2023-03-08 11:53:55,992 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:53:55,992 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:55,993 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:55,993 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:55,993 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,993 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:53:55,993 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:53:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,995 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:55,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,996 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:55,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:55,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:55,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:55,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:53:55,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35a57579[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:55,997 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:53:55,997 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,998 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:55,998 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:53:55,998 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:53:55,999 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:55,999 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:56,000 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:53:56,000 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,000 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:56,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:56,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:56,000 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:53:56,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:56,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:56,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:56,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:56,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:53:56,003 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:53:56,003 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:56,003 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:56,003 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,003 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:56,004 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,004 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:56,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:56,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,004 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:56,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:56,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:56,005 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:56,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,012 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:56,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,018 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,018 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:56,018 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:56,018 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:56,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:56,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:56,018 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:56,018 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:56,018 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,019 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:56,019 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:56,019 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,019 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:56,019 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:56,120 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:56,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:56,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:56,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:56,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,128 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:56,128 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:56,128 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:53:56,128 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:53:56,129 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,129 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,130 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:56,130 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:56,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:56,130 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,130 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:56,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:56,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,131 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:53:56,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,131 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:53:56,131 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:56,132 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:53:56,132 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:56,132 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:56,132 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:56,132 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. started... 2023-03-08 11:53:56,132 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 148f994b12b7c170e5fbb2fd36359918 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:56,192 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:53:56,548 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512012081f67fd297419b9174e0514faa196c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512012081f67fd297419b9174e0514faa196c 2023-03-08 11:53:56,954 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/d2778ef284d1432baf5335a7287b1e1d 2023-03-08 11:53:56,961 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/d2778ef284d1432baf5335a7287b1e1d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d2778ef284d1432baf5335a7287b1e1d 2023-03-08 11:53:56,968 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d2778ef284d1432baf5335a7287b1e1d, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 11:53:56,969 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 148f994b12b7c170e5fbb2fd36359918 in 837ms, sequenceid=13, compaction requested=true 2023-03-08 11:53:56,969 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:53:56,969 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:56,969 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:56,969 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:56,969 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,969 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:53:56,969 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:53:56,971 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:56,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:56,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:56,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:56,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:56,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:56,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:53:56,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e4ad750[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:56,973 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:53:56,973 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,975 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,976 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,976 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:56,976 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:53:56,976 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:53:56,976 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:56,977 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,977 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:53:56,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:56,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:56,977 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:56,977 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:53:56,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:56,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:56,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:56,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:56,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:53:56,981 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:53:56,981 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:56,981 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:56,981 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,981 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:56,982 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,982 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:56,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:56,982 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:56,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:56,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:56,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:56,983 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:56,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,984 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:56,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,986 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,987 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,987 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,987 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:56,987 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,987 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:56,987 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:56,987 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:56,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:56,988 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:56,988 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:56,992 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:56,992 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:56,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:56,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:56,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:56,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:57,088 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:57,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:57,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:57,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:57,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:57,095 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:57,095 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:57,096 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:53:57,096 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:53:57,096 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:57,097 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:57,099 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:57,099 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:57,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:57,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:57,099 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:57,099 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:57,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:57,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:57,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:57,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:57,100 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:57,100 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:53:57,100 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:57,100 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:53:57,100 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:57,101 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:53:57,101 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:57,101 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:57,101 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:57,101 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. started... 2023-03-08 11:53:57,101 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 148f994b12b7c170e5fbb2fd36359918 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:57,516 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120564a8a50f4ec64975b7b32ce543552643 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120564a8a50f4ec64975b7b32ce543552643 2023-03-08 11:53:57,923 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/59921f1c0ca048639d4be2ddbd6ef007 2023-03-08 11:53:57,931 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/59921f1c0ca048639d4be2ddbd6ef007 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/59921f1c0ca048639d4be2ddbd6ef007 2023-03-08 11:53:57,936 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/59921f1c0ca048639d4be2ddbd6ef007, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 11:53:57,937 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 148f994b12b7c170e5fbb2fd36359918 in 836ms, sequenceid=17, compaction requested=true 2023-03-08 11:53:57,937 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:53:57,938 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:57,938 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:57,938 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:57,938 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:57,938 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:53:57,938 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:53:57,999 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:57,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:57,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:57,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:57,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:57,999 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,000 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:58,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:58,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:58,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:58,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:53:58,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c39a6d0[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:58,001 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:53:58,002 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,045 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,045 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,046 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:58,046 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:53:58,046 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:53:58,046 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:58,130 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:53:58,130 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,130 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:58,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:58,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:58,130 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:53:58,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:58,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:58,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:58,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:53:58,133 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:53:58,133 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:58,133 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:58,133 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,133 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:58,176 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,176 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,176 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:58,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:58,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:58,177 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:58,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:58,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:58,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,178 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,222 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,222 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:58,222 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,222 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:58,222 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,223 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:58,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:58,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:58,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:58,222 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:58,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:53:58,223 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,224 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,224 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,224 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:58,224 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:58,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:58,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:58,324 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:58,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:58,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:58,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:58,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,332 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:58,332 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:58,332 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:53:58,332 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:53:58,333 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,333 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,368 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,368 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:58,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:58,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:58,369 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,369 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:58,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:58,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:53:58,371 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,371 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:53:58,371 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:58,371 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:53:58,371 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:58,372 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:58,372 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:58,372 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. started... 2023-03-08 11:53:58,372 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 148f994b12b7c170e5fbb2fd36359918 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:58,386 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228b4a910ec44fe4d1dbd60cd636ba59147 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228b4a910ec44fe4d1dbd60cd636ba59147 2023-03-08 11:53:58,792 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/1a010a1d1fa44112b0b8a45da03b0ee2 2023-03-08 11:53:58,799 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/1a010a1d1fa44112b0b8a45da03b0ee2 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/1a010a1d1fa44112b0b8a45da03b0ee2 2023-03-08 11:53:58,806 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/1a010a1d1fa44112b0b8a45da03b0ee2, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 11:53:58,807 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 148f994b12b7c170e5fbb2fd36359918 in 435ms, sequenceid=21, compaction requested=true 2023-03-08 11:53:58,807 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:53:58,807 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:58,807 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:58,807 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:58,807 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,807 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:53:58,807 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:53:58,809 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,809 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:58,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:58,809 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,809 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:58,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:58,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69a1e719[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:58,810 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:53:58,810 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,813 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,813 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:53:58,813 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:53:58,813 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:58,813 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,813 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:58,814 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:53:58,814 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,814 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:58,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:58,815 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:53:58,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:58,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:58,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:58,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:58,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:58,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:53:58,818 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:53:58,818 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:58,818 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:58,818 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,818 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:58,823 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,823 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,823 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:58,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:58,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:58,824 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:58,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:58,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:58,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:58,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,825 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:58,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,828 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,828 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,828 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:58,828 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:58,828 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,828 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,828 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,828 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,828 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:58,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:58,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:58,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:58,830 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:58,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:58,831 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:58,831 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:53:58,831 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:58,832 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:58,932 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:58,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:58,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:58,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:58,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,940 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:58,940 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:58,940 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:53:58,940 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:53:58,941 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,941 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,942 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,942 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:58,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:58,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:58,942 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:58,943 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:58,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:58,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,944 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,948 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:53:58,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:58,948 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:53:58,948 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:58,949 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:53:58,949 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:58,949 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:58,949 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:58,949 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. started... 2023-03-08 11:53:58,949 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 148f994b12b7c170e5fbb2fd36359918 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:59,371 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231886d503205034cc5a9f46a86649e0c23 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231886d503205034cc5a9f46a86649e0c23 2023-03-08 11:53:59,379 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/b46e2521c85746d481a98b50a6d060cd 2023-03-08 11:53:59,386 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/b46e2521c85746d481a98b50a6d060cd as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b46e2521c85746d481a98b50a6d060cd 2023-03-08 11:53:59,393 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b46e2521c85746d481a98b50a6d060cd, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 11:53:59,394 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 148f994b12b7c170e5fbb2fd36359918 in 445ms, sequenceid=25, compaction requested=true 2023-03-08 11:53:59,394 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:53:59,394 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:59,394 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:53:59,394 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:53:59,394 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,394 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:53:59,394 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:53:59,396 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,396 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:59,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:59,397 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,397 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:53:59,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:59,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:59,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:59,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:53:59,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@773149e9[Count = 0] remaining members to acquire global barrier 2023-03-08 11:53:59,399 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:53:59,399 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,401 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,401 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:53:59,401 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:53:59,401 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,401 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:53:59,401 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:53:59,402 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,402 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:53:59,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:59,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:59,402 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:59,403 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:53:59,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:59,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:59,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:59,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:53:59,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:53:59,405 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:53:59,405 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:53:59,406 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:53:59,406 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,406 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:53:59,407 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,407 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,407 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:59,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:53:59,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:53:59,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,408 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:59,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:53:59,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:59,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:53:59,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,409 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:53:59,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,411 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,411 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,411 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,411 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,411 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,411 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:59,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:53:59,411 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:59,411 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:59,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:59,411 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:53:59,412 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:53:59,411 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:53:59,412 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:53:59,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:53:59,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:53:59,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:53:59,513 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:53:59,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:53:59,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:53:59,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:53:59,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,520 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:53:59,520 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:59,521 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:53:59,521 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:53:59,521 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,521 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,523 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:53:59,523 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:53:59,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:53:59,524 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:53:59,524 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:53:59,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:53:59,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,525 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:53:59,525 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:53:59,525 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:53:59,525 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:53:59,525 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:53:59,526 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:53:59,526 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:53:59,526 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:53:59,526 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. started... 2023-03-08 11:53:59,526 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 148f994b12b7c170e5fbb2fd36359918 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:53:59,946 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601012bc38ea949b1417e9bf4985603cde726 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601012bc38ea949b1417e9bf4985603cde726 2023-03-08 11:54:00,355 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/d6898c75ac844b34a501d0a3ffebaeae 2023-03-08 11:54:00,361 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/d6898c75ac844b34a501d0a3ffebaeae as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d6898c75ac844b34a501d0a3ffebaeae 2023-03-08 11:54:00,366 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d6898c75ac844b34a501d0a3ffebaeae, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 11:54:00,368 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 148f994b12b7c170e5fbb2fd36359918 in 841ms, sequenceid=29, compaction requested=true 2023-03-08 11:54:00,368 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:54:00,368 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:54:00,368 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:00,368 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:00,368 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,368 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:54:00,368 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:54:00,369 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,369 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:00,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:00,370 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:00,370 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:00,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:00,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:00,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:54:00,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f89e008[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:00,371 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:00,371 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,373 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,373 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:00,373 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:54:00,373 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:00,373 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,373 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:00,374 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:54:00,374 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:00,374 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,374 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:54:00,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:00,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:00,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:00,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:00,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:00,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:00,378 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:00,378 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:54:00,378 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:00,378 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:00,378 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,378 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:00,380 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,380 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:00,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:00,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,380 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:00,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:00,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:00,381 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:00,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:00,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,383 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:00,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,386 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,386 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:00,386 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:00,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:00,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:54:00,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:00,386 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:00,386 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,386 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:00,386 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:00,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:54:00,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:00,387 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:00,387 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,387 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:00,388 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,388 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,488 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:00,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:00,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:00,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:00,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,498 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:54:00,498 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:00,499 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:54:00,499 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:00,499 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,499 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,501 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:00,501 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:00,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:00,501 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:00,501 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:00,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:00,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,502 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,502 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:54:00,502 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:00,504 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:54:00,504 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:00,504 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:54:00,504 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:00,504 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:00,505 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:54:00,505 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. started... 2023-03-08 11:54:00,505 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 148f994b12b7c170e5fbb2fd36359918 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:00,922 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010334dba3b4ea99478cba08ca9a59f227a8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010334dba3b4ea99478cba08ca9a59f227a8 2023-03-08 11:54:01,328 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/653354740793492493b003b6ea2808a3 2023-03-08 11:54:01,334 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/653354740793492493b003b6ea2808a3 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/653354740793492493b003b6ea2808a3 2023-03-08 11:54:01,340 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/653354740793492493b003b6ea2808a3, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 11:54:01,341 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 148f994b12b7c170e5fbb2fd36359918 in 836ms, sequenceid=33, compaction requested=true 2023-03-08 11:54:01,341 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:54:01,341 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:54:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:54:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:54:01,342 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,342 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:01,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:54:01,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ee7ce0f[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:01,344 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:01,344 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,346 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,346 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:01,346 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:54:01,346 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:01,346 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,346 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:01,350 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,350 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:54:01,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:01,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:01,350 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:01,350 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:01,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:01,353 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:54:01,353 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:01,353 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:01,353 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,353 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:01,354 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,354 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:01,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:01,355 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:01,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:01,355 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:01,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:01,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:01,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,356 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:01,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,359 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,359 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,359 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,360 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,360 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,360 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,360 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:01,359 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:01,360 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:54:01,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:01,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:01,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:54:01,360 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:01,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:01,361 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:01,361 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:01,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:01,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:01,461 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:01,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:01,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:01,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:01,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,469 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:54:01,469 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:01,469 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:54:01,469 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:01,470 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,470 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,471 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:01,471 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:01,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:01,471 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:01,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,471 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:01,472 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:01,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:54:01,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:01,472 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:54:01,472 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:01,473 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:54:01,473 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:01,473 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:01,473 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:54:01,473 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. started... 2023-03-08 11:54:01,473 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 148f994b12b7c170e5fbb2fd36359918 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:01,889 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509073b19ef4aad754abe9df2eafe511cb975 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509073b19ef4aad754abe9df2eafe511cb975 2023-03-08 11:54:02,295 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/085aefdc5b2c4fffb828ce8b763216fb 2023-03-08 11:54:02,301 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/085aefdc5b2c4fffb828ce8b763216fb as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/085aefdc5b2c4fffb828ce8b763216fb 2023-03-08 11:54:02,306 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/085aefdc5b2c4fffb828ce8b763216fb, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 11:54:02,307 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 148f994b12b7c170e5fbb2fd36359918 in 834ms, sequenceid=37, compaction requested=true 2023-03-08 11:54:02,307 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:54:02,307 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:54:02,307 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:02,307 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:02,307 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,307 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:54:02,307 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:54:02,309 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,309 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:02,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:02,309 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,309 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:02,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:02,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:02,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:02,310 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:54:02,310 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@389873e5[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:02,311 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:02,311 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,312 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,312 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:02,312 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:54:02,312 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:02,312 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,312 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:02,315 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:54:02,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:02,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:02,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:02,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:54:02,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:02,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:02,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:02,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:02,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:02,318 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:54:02,318 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:02,318 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:02,318 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,318 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:02,320 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,320 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:02,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:02,320 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:02,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,320 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:02,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:02,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:02,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:02,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,321 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:02,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,324 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,324 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,324 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,324 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,324 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,324 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,324 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:02,324 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:54:02,324 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:02,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:54:02,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:02,324 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:02,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:02,324 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:02,325 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:02,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:02,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:02,325 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:02,425 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:02,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:02,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:02,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:02,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,432 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:54:02,433 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:02,433 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:54:02,433 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:02,433 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,434 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,435 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,435 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:02,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:02,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:02,435 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:02,435 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:02,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,436 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,436 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,436 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:54:02,436 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:02,436 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:54:02,436 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:02,437 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:54:02,438 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:02,438 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:02,438 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:54:02,438 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. started... 2023-03-08 11:54:02,438 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 148f994b12b7c170e5fbb2fd36359918 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:02,852 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e0f4dcd53cfe413a82938e1ad79f05e3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120e0f4dcd53cfe413a82938e1ad79f05e3 2023-03-08 11:54:03,258 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/801fc5776c93470fb48b612323ee5570 2023-03-08 11:54:03,265 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/801fc5776c93470fb48b612323ee5570 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/801fc5776c93470fb48b612323ee5570 2023-03-08 11:54:03,270 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/801fc5776c93470fb48b612323ee5570, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 11:54:03,271 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 148f994b12b7c170e5fbb2fd36359918 in 832ms, sequenceid=41, compaction requested=true 2023-03-08 11:54:03,271 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:54:03,271 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:54:03,271 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:03,271 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:03,271 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,271 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:54:03,271 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:54:03,273 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,273 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:03,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:03,274 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,274 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:03,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:03,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:03,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:03,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:54:03,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@451b1b1e[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:03,275 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:03,275 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,276 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,276 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:03,276 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:54:03,276 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:03,276 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,276 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:03,278 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,278 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:54:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:03,278 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:03,278 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:54:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:03,280 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:03,280 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:54:03,281 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:03,281 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:03,281 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,281 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:03,282 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,282 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:03,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,282 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:03,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:03,282 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,283 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:03,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,290 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,290 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:03,290 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:03,290 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:03,290 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:03,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:03,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 11:54:03,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:03,290 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,291 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:03,291 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,291 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,291 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:03,292 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:03,292 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:03,392 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:03,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:03,394 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-08 11:54:03,396 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:54:03,397 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:54:03,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:03,398 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277043398 2023-03-08 11:54:03,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:54:03,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-08 11:54:03,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:54:03,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x02f847fd to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:54:03,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7549b8fe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:54:03,406 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:54:03,843 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103216243f6e0364d0e85afaade76dca892 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103216243f6e0364d0e85afaade76dca892 2023-03-08 11:54:03,844 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:54:03,857 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:54:03,858 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57306, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:54:03,864 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:54:03,865 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36492, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:54:03,875 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1e58982e23eb441d8df4eba982902f28 first=Optional[r5] last=Optional[r6] 2023-03-08 11:54:03,876 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1e58982e23eb441d8df4eba982902f28}] 2023-03-08 11:54:03,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1e58982e23eb441d8df4eba982902f28 for inclusion in 148f994b12b7c170e5fbb2fd36359918/family1 2023-03-08 11:54:03,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-08 11:54:03,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:54:03,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:54:03,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1e58982e23eb441d8df4eba982902f28 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__m59p1dbpb1c81a915kn163t516rai5ab8hs3506e5ug556p9ui4ld9mqk2k2pba3/family1/1e58982e23eb441d8df4eba982902f28 2023-03-08 11:54:03,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__m59p1dbpb1c81a915kn163t516rai5ab8hs3506e5ug556p9ui4ld9mqk2k2pba3/family1/1e58982e23eb441d8df4eba982902f28 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b413923fe3234683aca3ca6c0c14f9c5_SeqId_44_ 2023-03-08 11:54:03,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__m59p1dbpb1c81a915kn163t516rai5ab8hs3506e5ug556p9ui4ld9mqk2k2pba3/family1/1e58982e23eb441d8df4eba982902f28 into 148f994b12b7c170e5fbb2fd36359918/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b413923fe3234683aca3ca6c0c14f9c5_SeqId_44_ - updating store file list. 2023-03-08 11:54:03,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b413923fe3234683aca3ca6c0c14f9c5_SeqId_44_ into 148f994b12b7c170e5fbb2fd36359918/family1 2023-03-08 11:54:03,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__m59p1dbpb1c81a915kn163t516rai5ab8hs3506e5ug556p9ui4ld9mqk2k2pba3/family1/1e58982e23eb441d8df4eba982902f28 into 148f994b12b7c170e5fbb2fd36359918/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b413923fe3234683aca3ca6c0c14f9c5_SeqId_44_) 2023-03-08 11:54:03,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__m59p1dbpb1c81a915kn163t516rai5ab8hs3506e5ug556p9ui4ld9mqk2k2pba3/family1/1e58982e23eb441d8df4eba982902f28 2023-03-08 11:54:03,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 11:54:03,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 11:54:03,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 148f994b12b7c170e5fbb2fd36359918/family1 is initiating minor compaction 2023-03-08 11:54:03,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@43f03d4b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:03,896 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 148f994b12b7c170e5fbb2fd36359918/family1 in testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:54:03,896 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/aa1ec47c53dd487282bf194c9e54f245, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d2778ef284d1432baf5335a7287b1e1d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/59921f1c0ca048639d4be2ddbd6ef007, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/1a010a1d1fa44112b0b8a45da03b0ee2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b46e2521c85746d481a98b50a6d060cd, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d6898c75ac844b34a501d0a3ffebaeae, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/653354740793492493b003b6ea2808a3, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/085aefdc5b2c4fffb828ce8b763216fb, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/801fc5776c93470fb48b612323ee5570, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b413923fe3234683aca3ca6c0c14f9c5_SeqId_44_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp, totalSize=49.7 K 2023-03-08 11:54:03,896 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting aa1ec47c53dd487282bf194c9e54f245, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 11:54:03,896 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting d2778ef284d1432baf5335a7287b1e1d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 11:54:03,896 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 59921f1c0ca048639d4be2ddbd6ef007, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 11:54:03,897 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 1a010a1d1fa44112b0b8a45da03b0ee2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 11:54:03,897 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting b46e2521c85746d481a98b50a6d060cd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 11:54:03,897 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting d6898c75ac844b34a501d0a3ffebaeae, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 11:54:03,897 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 653354740793492493b003b6ea2808a3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 11:54:03,897 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 085aefdc5b2c4fffb828ce8b763216fb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 11:54:03,897 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 801fc5776c93470fb48b612323ee5570, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 11:54:03,897 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting b413923fe3234683aca3ca6c0c14f9c5_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 11:54:03,898 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:54:03,899 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:03,901 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601012bc38ea949b1417e9bf4985603cde726 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601012bc38ea949b1417e9bf4985603cde726 2023-03-08 11:54:03,902 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010334dba3b4ea99478cba08ca9a59f227a8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010334dba3b4ea99478cba08ca9a59f227a8 2023-03-08 11:54:03,902 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-08 11:54:03,902 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-08 11:54:03,903 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:54:03,903 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:54:03,931 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 148f994b12b7c170e5fbb2fd36359918#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:54:04,750 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/c7858134642e4b439b1afb99b54e81b2 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/c7858134642e4b439b1afb99b54e81b2 2023-03-08 11:54:04,753 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231065a930185184beea1ec8fd5bfea7ced to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231065a930185184beea1ec8fd5bfea7ced 2023-03-08 11:54:04,754 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:54:04,757 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 148f994b12b7c170e5fbb2fd36359918/family1 of 148f994b12b7c170e5fbb2fd36359918 into c7858134642e4b439b1afb99b54e81b2(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:54:04,757 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:54:04,757 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678276443896; duration=0sec 2023-03-08 11:54:04,757 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:04,768 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/aa1ec47c53dd487282bf194c9e54f245, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d2778ef284d1432baf5335a7287b1e1d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/59921f1c0ca048639d4be2ddbd6ef007, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/1a010a1d1fa44112b0b8a45da03b0ee2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b46e2521c85746d481a98b50a6d060cd, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d6898c75ac844b34a501d0a3ffebaeae, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/653354740793492493b003b6ea2808a3, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/085aefdc5b2c4fffb828ce8b763216fb, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/801fc5776c93470fb48b612323ee5570, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b413923fe3234683aca3ca6c0c14f9c5_SeqId_44_] to archive 2023-03-08 11:54:04,769 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:04,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/aa1ec47c53dd487282bf194c9e54f245 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/aa1ec47c53dd487282bf194c9e54f245 2023-03-08 11:54:04,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d2778ef284d1432baf5335a7287b1e1d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d2778ef284d1432baf5335a7287b1e1d 2023-03-08 11:54:04,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/59921f1c0ca048639d4be2ddbd6ef007 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/59921f1c0ca048639d4be2ddbd6ef007 2023-03-08 11:54:04,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/1a010a1d1fa44112b0b8a45da03b0ee2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/1a010a1d1fa44112b0b8a45da03b0ee2 2023-03-08 11:54:04,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b46e2521c85746d481a98b50a6d060cd to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b46e2521c85746d481a98b50a6d060cd 2023-03-08 11:54:04,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d6898c75ac844b34a501d0a3ffebaeae to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/d6898c75ac844b34a501d0a3ffebaeae 2023-03-08 11:54:04,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/653354740793492493b003b6ea2808a3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/653354740793492493b003b6ea2808a3 2023-03-08 11:54:04,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/085aefdc5b2c4fffb828ce8b763216fb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/085aefdc5b2c4fffb828ce8b763216fb 2023-03-08 11:54:04,783 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/801fc5776c93470fb48b612323ee5570 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/801fc5776c93470fb48b612323ee5570 2023-03-08 11:54:04,786 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b413923fe3234683aca3ca6c0c14f9c5_SeqId_44_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/b413923fe3234683aca3ca6c0c14f9c5_SeqId_44_ 2023-03-08 11:54:04,788 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9134350d2b944491a2c02998e5e4c430 first=Optional[r1] last=Optional[r4] 2023-03-08 11:54:04,789 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9134350d2b944491a2c02998e5e4c430}] 2023-03-08 11:54:04,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9134350d2b944491a2c02998e5e4c430 for inclusion in 148f994b12b7c170e5fbb2fd36359918/family1 2023-03-08 11:54:04,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-08 11:54:04,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:54:04,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:54:04,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9134350d2b944491a2c02998e5e4c430 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4g82m292jj471cdpb3eppu30pkpa6vddu948i4q2kf9cb7lttjj1desb31lugddm/family1/9134350d2b944491a2c02998e5e4c430 2023-03-08 11:54:04,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4g82m292jj471cdpb3eppu30pkpa6vddu948i4q2kf9cb7lttjj1desb31lugddm/family1/9134350d2b944491a2c02998e5e4c430 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/fda23fc09fae43b4b26ccfe31f707174_SeqId_47_ 2023-03-08 11:54:04,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4g82m292jj471cdpb3eppu30pkpa6vddu948i4q2kf9cb7lttjj1desb31lugddm/family1/9134350d2b944491a2c02998e5e4c430 into 148f994b12b7c170e5fbb2fd36359918/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/fda23fc09fae43b4b26ccfe31f707174_SeqId_47_ - updating store file list. 2023-03-08 11:54:04,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/fda23fc09fae43b4b26ccfe31f707174_SeqId_47_ into 148f994b12b7c170e5fbb2fd36359918/family1 2023-03-08 11:54:04,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4g82m292jj471cdpb3eppu30pkpa6vddu948i4q2kf9cb7lttjj1desb31lugddm/family1/9134350d2b944491a2c02998e5e4c430 into 148f994b12b7c170e5fbb2fd36359918/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/fda23fc09fae43b4b26ccfe31f707174_SeqId_47_) 2023-03-08 11:54:04,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4g82m292jj471cdpb3eppu30pkpa6vddu948i4q2kf9cb7lttjj1desb31lugddm/family1/9134350d2b944491a2c02998e5e4c430 2023-03-08 11:54:04,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 11:54:04,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 11:54:04,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 148f994b12b7c170e5fbb2fd36359918/family1 is initiating minor compaction (all files) 2023-03-08 11:54:04,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@42b4dac; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:04,804 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 148f994b12b7c170e5fbb2fd36359918/family1 in testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:54:04,804 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/358914a0885a4a3d88cb093aa0c91025, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/c7858134642e4b439b1afb99b54e81b2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/fda23fc09fae43b4b26ccfe31f707174_SeqId_47_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp, totalSize=178.1 K 2023-03-08 11:54:04,805 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 358914a0885a4a3d88cb093aa0c91025, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838541 2023-03-08 11:54:04,805 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting c7858134642e4b439b1afb99b54e81b2, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238541 2023-03-08 11:54:04,806 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting fda23fc09fae43b4b26ccfe31f707174_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965238541 2023-03-08 11:54:04,806 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:54:04,807 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:04,809 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512012081f67fd297419b9174e0514faa196c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512012081f67fd297419b9174e0514faa196c 2023-03-08 11:54:04,810 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120564a8a50f4ec64975b7b32ce543552643 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120564a8a50f4ec64975b7b32ce543552643 2023-03-08 11:54:04,812 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228b4a910ec44fe4d1dbd60cd636ba59147 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228b4a910ec44fe4d1dbd60cd636ba59147 2023-03-08 11:54:04,813 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231886d503205034cc5a9f46a86649e0c23 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231886d503205034cc5a9f46a86649e0c23 2023-03-08 11:54:04,813 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-08 11:54:04,813 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-08 11:54:04,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:54:04,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:54:04,823 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 148f994b12b7c170e5fbb2fd36359918#family1#compaction#23 average throughput is 19.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:54:05,665 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511306ad7a80a4ba04d0c9476c8c7c61ca30d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511306ad7a80a4ba04d0c9476c8c7c61ca30d 2023-03-08 11:54:05,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:54:05,691 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c3ba8cdb382e4e629135f8201f6c6dbd first=Optional[r00] last=Optional[r8] 2023-03-08 11:54:05,692 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c3ba8cdb382e4e629135f8201f6c6dbd}] 2023-03-08 11:54:05,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c3ba8cdb382e4e629135f8201f6c6dbd for inclusion in 148f994b12b7c170e5fbb2fd36359918/family1 2023-03-08 11:54:05,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-08 11:54:05,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:54:05,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:54:05,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c3ba8cdb382e4e629135f8201f6c6dbd to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__82jvi9q4qhqa9fottj7hae2s1ru76gtcui6b1pjet7ptjlite2s2kcfr2d0sbn57/family1/c3ba8cdb382e4e629135f8201f6c6dbd 2023-03-08 11:54:05,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__82jvi9q4qhqa9fottj7hae2s1ru76gtcui6b1pjet7ptjlite2s2kcfr2d0sbn57/family1/c3ba8cdb382e4e629135f8201f6c6dbd as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/5812da11946748d8801c2a42286eff4d_SeqId_49_ 2023-03-08 11:54:05,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__82jvi9q4qhqa9fottj7hae2s1ru76gtcui6b1pjet7ptjlite2s2kcfr2d0sbn57/family1/c3ba8cdb382e4e629135f8201f6c6dbd into 148f994b12b7c170e5fbb2fd36359918/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/5812da11946748d8801c2a42286eff4d_SeqId_49_ - updating store file list. 2023-03-08 11:54:05,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/5812da11946748d8801c2a42286eff4d_SeqId_49_ into 148f994b12b7c170e5fbb2fd36359918/family1 2023-03-08 11:54:05,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__82jvi9q4qhqa9fottj7hae2s1ru76gtcui6b1pjet7ptjlite2s2kcfr2d0sbn57/family1/c3ba8cdb382e4e629135f8201f6c6dbd into 148f994b12b7c170e5fbb2fd36359918/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/5812da11946748d8801c2a42286eff4d_SeqId_49_) 2023-03-08 11:54:05,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__82jvi9q4qhqa9fottj7hae2s1ru76gtcui6b1pjet7ptjlite2s2kcfr2d0sbn57/family1/c3ba8cdb382e4e629135f8201f6c6dbd 2023-03-08 11:54:05,707 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:54:05,708 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:05,710 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120e0f4dcd53cfe413a82938e1ad79f05e3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120e0f4dcd53cfe413a82938e1ad79f05e3 2023-03-08 11:54:05,711 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128eeeaa333ff214e029853f6e0b762138e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128eeeaa333ff214e029853f6e0b762138e 2023-03-08 11:54:05,712 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113091abce7b0aa547c08961bf51004bbe45 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113091abce7b0aa547c08961bf51004bbe45 2023-03-08 11:54:05,712 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-08 11:54:05,712 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 11:54:05,712 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:54:05,712 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:54:05,712 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:54:05,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:54:05,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:05,713 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:54:05,715 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.3180 sec 2023-03-08 11:54:05,815 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=254 (was 250) Potentially hanging thread: hconnection-0x3ea99669-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3ea99669-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:38930 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:38976 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/c99c740b04db48d5a78076ca8d8ecb5e java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:39000 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:39024 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3ea99669-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:39032 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3ea99669-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_735287611_15 at /127.0.0.1:38888 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51863@0x02f847fd sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1824476703.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/cluster_7d11bbd1-2ad2-65f1-e179-3d37987b50d1/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:38878 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51863@0x02f847fd-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:38966 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:40792 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51863@0x02f847fd-SendThread(127.0.0.1:51863) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:39100 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:38940 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:38916 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:38882 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:40770 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3ea99669-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_271635103_15 at /127.0.0.1:40746 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3ea99669-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3ea99669-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_735287611_15 at /127.0.0.1:38920 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=608 (was 593) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=198 (was 234), ProcessCount=170 (was 170), AvailableMemoryMB=8075 (was 7455) - AvailableMemoryMB LEAK? - 2023-03-08 11:54:05,827 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=254, OpenFileDescriptor=608, MaxFileDescriptor=60000, SystemLoadAverage=198, ProcessCount=170, AvailableMemoryMB=8074 2023-03-08 11:54:05,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 11:54:05,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:05,832 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:54:05,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-08 11:54:05,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 11:54:05,833 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:54:05,835 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b 2023-03-08 11:54:05,836 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b empty. 2023-03-08 11:54:05,836 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b 2023-03-08 11:54:05,836 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-08 11:54:05,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 11:54:06,053 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/.tmp/family1/c99c740b04db48d5a78076ca8d8ecb5e as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/c99c740b04db48d5a78076ca8d8ecb5e 2023-03-08 11:54:06,059 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 148f994b12b7c170e5fbb2fd36359918/family1 of 148f994b12b7c170e5fbb2fd36359918 into c99c740b04db48d5a78076ca8d8ecb5e(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:54:06,059 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:54:06,059 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678276444804; duration=1sec 2023-03-08 11:54:06,059 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:06,061 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/358914a0885a4a3d88cb093aa0c91025, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/c7858134642e4b439b1afb99b54e81b2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/fda23fc09fae43b4b26ccfe31f707174_SeqId_47_] to archive 2023-03-08 11:54:06,062 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:06,064 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/358914a0885a4a3d88cb093aa0c91025 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/358914a0885a4a3d88cb093aa0c91025 2023-03-08 11:54:06,066 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/c7858134642e4b439b1afb99b54e81b2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/c7858134642e4b439b1afb99b54e81b2 2023-03-08 11:54:06,067 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/fda23fc09fae43b4b26ccfe31f707174_SeqId_47_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/family1/fda23fc09fae43b4b26ccfe31f707174_SeqId_47_ 2023-03-08 11:54:06,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 11:54:06,246 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 11:54:06,247 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => c154d984bcb433ac75837c927dc4662b, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:54:06,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 11:54:06,654 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:06,654 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing c154d984bcb433ac75837c927dc4662b, disabling compactions & flushes 2023-03-08 11:54:06,654 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:06,654 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:06,654 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. after waiting 0 ms 2023-03-08 11:54:06,654 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:06,654 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:06,654 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:06,657 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:54:06,657 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678276446657"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276446657"}]},"ts":"1678276446657"} 2023-03-08 11:54:06,659 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 11:54:06,660 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:54:06,660 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276446660"}]},"ts":"1678276446660"} 2023-03-08 11:54:06,661 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-08 11:54:06,667 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c154d984bcb433ac75837c927dc4662b, ASSIGN}] 2023-03-08 11:54:06,669 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c154d984bcb433ac75837c927dc4662b, ASSIGN 2023-03-08 11:54:06,670 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c154d984bcb433ac75837c927dc4662b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:54:06,821 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=c154d984bcb433ac75837c927dc4662b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:06,821 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678276446821"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276446821"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276446821"}]},"ts":"1678276446821"} 2023-03-08 11:54:06,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure c154d984bcb433ac75837c927dc4662b, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:06,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 11:54:06,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:06,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c154d984bcb433ac75837c927dc4662b, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:54:06,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy c154d984bcb433ac75837c927dc4662b 2023-03-08 11:54:06,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:06,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c154d984bcb433ac75837c927dc4662b 2023-03-08 11:54:06,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c154d984bcb433ac75837c927dc4662b 2023-03-08 11:54:06,980 INFO [StoreOpener-c154d984bcb433ac75837c927dc4662b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c154d984bcb433ac75837c927dc4662b 2023-03-08 11:54:06,981 DEBUG [StoreOpener-c154d984bcb433ac75837c927dc4662b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1 2023-03-08 11:54:06,981 DEBUG [StoreOpener-c154d984bcb433ac75837c927dc4662b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1 2023-03-08 11:54:06,982 INFO [StoreOpener-c154d984bcb433ac75837c927dc4662b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c154d984bcb433ac75837c927dc4662b columnFamilyName family1 2023-03-08 11:54:06,983 INFO [StoreOpener-c154d984bcb433ac75837c927dc4662b-1] regionserver.HStore(310): Store=c154d984bcb433ac75837c927dc4662b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:06,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b 2023-03-08 11:54:06,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b 2023-03-08 11:54:06,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c154d984bcb433ac75837c927dc4662b 2023-03-08 11:54:06,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:54:06,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c154d984bcb433ac75837c927dc4662b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64113352, jitterRate=-0.04463660717010498}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:54:06,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:06,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b., pid=25, masterSystemTime=1678276446975 2023-03-08 11:54:06,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:06,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:06,992 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=c154d984bcb433ac75837c927dc4662b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:06,992 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678276446992"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276446992"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276446992"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276446992"}]},"ts":"1678276446992"} 2023-03-08 11:54:06,995 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-08 11:54:06,996 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure c154d984bcb433ac75837c927dc4662b, server=jenkins-hbase4.apache.org,45551,1678276414946 in 171 msec 2023-03-08 11:54:06,997 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-08 11:54:06,997 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c154d984bcb433ac75837c927dc4662b, ASSIGN in 329 msec 2023-03-08 11:54:06,998 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:54:06,998 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276446998"}]},"ts":"1678276446998"} 2023-03-08 11:54:06,999 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-08 11:54:07,002 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:54:07,003 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1720 sec 2023-03-08 11:54:07,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 11:54:07,939 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-08 11:54:07,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:07,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:07,968 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:07,968 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:07,968 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:07,968 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:07,968 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:07,968 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:07,969 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:07,969 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:07,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:07,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:07,970 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:07,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:07,970 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:07,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:07,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:07,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:07,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:07,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:07,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:07,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:07,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:07,971 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:07,971 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. started... 2023-03-08 11:54:07,972 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c154d984bcb433ac75837c927dc4662b 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 11:54:08,393 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130c8d8d06e577840f99f342628239b475c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130c8d8d06e577840f99f342628239b475c 2023-03-08 11:54:08,797 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/9e0276541a924d35b654271eed911304 2023-03-08 11:54:08,803 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/9e0276541a924d35b654271eed911304 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/9e0276541a924d35b654271eed911304 2023-03-08 11:54:08,808 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/9e0276541a924d35b654271eed911304, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-08 11:54:08,809 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for c154d984bcb433ac75837c927dc4662b in 837ms, sequenceid=5, compaction requested=false 2023-03-08 11:54:08,809 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:08,810 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:08,810 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:08,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:08,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:08,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:08,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:08,812 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,812 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:08,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:08,812 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,812 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:08,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:08,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:08,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:08,813 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:08,814 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35e23a15[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:08,814 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:08,814 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,815 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,815 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:08,815 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:08,815 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,815 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:08,815 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:08,816 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,816 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:08,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:08,816 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:08,817 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:08,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:08,819 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:08,819 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:08,819 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:08,819 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,819 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:08,821 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,821 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:08,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:08,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,821 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:08,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:08,821 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:08,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:08,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:08,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,822 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:08,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,824 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:08,824 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,825 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:08,825 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,825 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,825 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,825 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,825 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,825 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,825 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:08,825 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:08,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:08,825 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:08,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:08,825 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:08,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:08,825 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:08,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:08,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:08,926 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:08,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:08,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:08,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:08,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,933 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:08,933 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:08,933 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:08,933 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:08,933 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,933 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,934 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:08,934 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:08,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:08,935 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:08,935 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:08,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:08,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,936 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:08,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:08,936 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:08,936 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. started... 2023-03-08 11:54:08,936 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c154d984bcb433ac75837c927dc4662b 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 11:54:09,349 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128d367a72003284c8289e017807b17a670 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128d367a72003284c8289e017807b17a670 2023-03-08 11:54:09,755 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/4cc5b7cc14c946f1a71764a5aecbccd5 2023-03-08 11:54:09,760 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/4cc5b7cc14c946f1a71764a5aecbccd5 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/4cc5b7cc14c946f1a71764a5aecbccd5 2023-03-08 11:54:09,765 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/4cc5b7cc14c946f1a71764a5aecbccd5, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 11:54:09,766 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for c154d984bcb433ac75837c927dc4662b in 829ms, sequenceid=9, compaction requested=false 2023-03-08 11:54:09,766 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:09,766 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,767 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:09,768 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,768 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:09,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:09,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55151d10[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:09,769 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:09,769 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,771 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,771 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:09,771 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,772 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:09,771 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:09,772 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:09,773 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:09,773 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,773 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:09,773 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:09,775 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:09,775 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:09,775 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:09,775 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:09,775 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,775 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:09,776 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,776 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:09,776 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:09,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,777 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:09,778 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,780 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,780 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,780 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:09,780 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:09,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:09,780 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:09,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:09,780 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:09,780 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:09,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:09,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:09,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:09,781 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:09,780 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,781 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:09,781 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,781 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,881 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:09,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:09,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:09,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:09,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,888 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:09,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:09,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:09,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:09,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,891 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:09,891 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:09,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:09,891 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:09,891 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:09,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,892 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:09,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,892 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,892 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:09,892 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:09,892 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:09,892 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:09,892 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:09,893 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:09,893 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:09,893 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:09,893 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. started... 2023-03-08 11:54:09,893 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c154d984bcb433ac75837c927dc4662b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:10,307 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120178a68c41550e475f8b4046caebe4fcc1 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120178a68c41550e475f8b4046caebe4fcc1 2023-03-08 11:54:10,712 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/a1e07ad8e735439684d333a32628f2c9 2023-03-08 11:54:10,718 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/a1e07ad8e735439684d333a32628f2c9 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/a1e07ad8e735439684d333a32628f2c9 2023-03-08 11:54:10,723 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/a1e07ad8e735439684d333a32628f2c9, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 11:54:10,724 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c154d984bcb433ac75837c927dc4662b in 831ms, sequenceid=13, compaction requested=true 2023-03-08 11:54:10,724 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:10,724 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:10,724 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:10,724 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:10,724 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,724 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:10,724 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:10,725 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,725 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:10,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:10,726 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,726 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:10,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:10,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:10,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:10,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:10,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e694b0c[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:10,727 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:10,727 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,728 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,728 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:10,728 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:10,728 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:10,728 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,728 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:10,729 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,729 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:10,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:10,730 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:10,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:10,730 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:10,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:10,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:10,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:10,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:10,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:10,732 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:10,732 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:10,732 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:10,732 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,732 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:10,734 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,734 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,734 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:10,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:10,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:10,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:10,734 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:10,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:10,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:10,735 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:10,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:10,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:10,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:10,740 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:10,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:10,740 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:10,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:10,841 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:10,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:10,846 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:10,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:10,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,848 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:10,848 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:10,848 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:10,848 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:10,848 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,848 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,849 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:10,849 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:10,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:10,850 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:10,850 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:10,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,850 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:10,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,850 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,851 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:10,851 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:10,851 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:10,851 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:10,851 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:10,851 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:10,851 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:10,851 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:10,851 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. started... 2023-03-08 11:54:10,851 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c154d984bcb433ac75837c927dc4662b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:11,264 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205a156faab4fbb47a092998b0f1c3e47dd to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205a156faab4fbb47a092998b0f1c3e47dd 2023-03-08 11:54:11,670 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/19c45b36a9fc46378acf169423e7d6a8 2023-03-08 11:54:11,675 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/19c45b36a9fc46378acf169423e7d6a8 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/19c45b36a9fc46378acf169423e7d6a8 2023-03-08 11:54:11,680 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/19c45b36a9fc46378acf169423e7d6a8, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 11:54:11,680 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c154d984bcb433ac75837c927dc4662b in 829ms, sequenceid=17, compaction requested=true 2023-03-08 11:54:11,681 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:11,681 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:11,681 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:11,681 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:11,681 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,681 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:11,681 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:11,683 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,683 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:11,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:11,683 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,683 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:11,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:11,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:11,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:11,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:11,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33376634[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:11,684 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:11,684 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,685 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,686 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:11,686 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:11,686 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:11,686 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,686 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:11,687 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:11,687 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,687 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:11,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:11,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:11,687 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:11,689 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:11,689 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:11,689 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:11,689 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:11,689 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,689 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:11,691 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,691 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,691 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:11,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:11,691 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:11,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,692 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:11,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:11,695 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:11,695 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:11,695 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:11,695 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:11,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:11,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:11,796 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:11,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:11,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:11,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:11,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,802 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:11,802 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:11,802 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:11,802 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:11,803 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,803 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,804 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,804 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:11,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:11,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:11,804 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:11,804 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:11,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:11,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,805 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,805 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:11,805 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:11,805 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:11,805 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:11,805 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:11,806 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:11,806 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:11,806 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:11,806 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. started... 2023-03-08 11:54:11,806 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c154d984bcb433ac75837c927dc4662b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:12,219 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c8e6ce38f2c149ef8bfbe59f7eff5e5f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228c8e6ce38f2c149ef8bfbe59f7eff5e5f 2023-03-08 11:54:12,503 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:54:12,625 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/b24732e0f01b4f8c8b32ecec58177445 2023-03-08 11:54:12,630 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/b24732e0f01b4f8c8b32ecec58177445 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/b24732e0f01b4f8c8b32ecec58177445 2023-03-08 11:54:12,635 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/b24732e0f01b4f8c8b32ecec58177445, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 11:54:12,636 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c154d984bcb433ac75837c927dc4662b in 830ms, sequenceid=21, compaction requested=true 2023-03-08 11:54:12,636 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:12,636 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:12,636 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:12,637 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:12,637 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,637 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:12,637 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:12,638 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,638 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:12,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:12,638 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,639 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:12,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:12,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1721d0d5[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:12,640 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:12,640 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,641 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,641 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:12,641 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:12,641 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:12,641 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,641 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:12,643 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,643 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:12,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:12,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:12,643 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:12,644 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:12,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:12,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:12,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:12,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:12,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:12,646 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:12,646 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:12,646 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:12,646 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,646 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:12,647 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,647 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:12,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:12,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,647 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:12,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:12,648 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:12,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:12,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:12,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,649 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:12,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,651 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,651 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:12,651 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,651 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:12,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:12,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:12,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:12,651 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:12,651 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:12,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:12,651 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,651 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:12,652 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:12,652 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,652 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:12,652 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:12,752 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:12,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:12,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:12,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:12,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,760 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:12,760 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:12,760 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:12,760 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:12,760 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,760 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,762 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,762 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:12,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:12,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:12,762 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:12,762 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:12,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:12,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,763 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,763 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:12,763 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:12,763 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:12,763 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:12,763 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:12,764 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:12,764 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:12,764 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:12,764 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. started... 2023-03-08 11:54:12,764 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c154d984bcb433ac75837c927dc4662b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:13,179 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311d89783d170c41b49d26b73c0c32cbb0 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512311d89783d170c41b49d26b73c0c32cbb0 2023-03-08 11:54:13,586 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/90a267d0035b42869ec6c0bccd4c0272 2023-03-08 11:54:13,592 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/90a267d0035b42869ec6c0bccd4c0272 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/90a267d0035b42869ec6c0bccd4c0272 2023-03-08 11:54:13,597 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/90a267d0035b42869ec6c0bccd4c0272, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 11:54:13,598 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c154d984bcb433ac75837c927dc4662b in 834ms, sequenceid=25, compaction requested=true 2023-03-08 11:54:13,598 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:13,599 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:13,599 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:13,599 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:13,599 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,599 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:13,599 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:13,602 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,602 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:13,602 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:13,602 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:13,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:13,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:13,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a21a2bc[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:13,604 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:13,604 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,605 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,605 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:13,605 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:13,605 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:13,605 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,605 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:13,606 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,606 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:13,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:13,606 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:13,607 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:13,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:13,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:13,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:13,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:13,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:13,609 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:13,609 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:13,609 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:13,609 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,609 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:13,614 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,614 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:13,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:13,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,614 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:13,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:13,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:13,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:13,614 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:13,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,616 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:13,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,618 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,618 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:13,618 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:13,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:13,618 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:13,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:13,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:13,618 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:13,618 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:13,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:13,618 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,618 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:13,619 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:13,618 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,619 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,619 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,719 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:13,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:13,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:13,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:13,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,728 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:13,728 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:13,728 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:13,728 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:13,729 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,729 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,730 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:13,730 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:13,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:13,730 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:13,730 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:13,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:13,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,731 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,732 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:13,732 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:13,732 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:13,733 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. started... 2023-03-08 11:54:13,733 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c154d984bcb433ac75837c927dc4662b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:14,148 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101d9ebb3767c254870b37870a687af2d30 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101d9ebb3767c254870b37870a687af2d30 2023-03-08 11:54:14,154 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/76fc979161c844f5a5dc838b425f754e 2023-03-08 11:54:14,159 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/76fc979161c844f5a5dc838b425f754e as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/76fc979161c844f5a5dc838b425f754e 2023-03-08 11:54:14,165 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/76fc979161c844f5a5dc838b425f754e, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 11:54:14,166 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c154d984bcb433ac75837c927dc4662b in 433ms, sequenceid=29, compaction requested=true 2023-03-08 11:54:14,166 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:14,166 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:14,166 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:14,166 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:14,166 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,166 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:14,166 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:14,168 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,168 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:14,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:14,169 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:14,169 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:14,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:14,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:14,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:14,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e74b5df[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:14,170 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:14,170 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,172 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,172 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:14,172 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:14,172 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:14,172 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,172 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:14,173 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:14,173 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,173 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:14,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:14,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:14,173 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:14,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:14,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:14,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:14,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:14,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:14,176 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:14,176 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:14,176 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:14,176 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,176 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:14,178 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,178 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:14,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:14,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,178 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:14,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:14,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:14,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:14,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,179 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:14,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,180 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:14,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:14,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,184 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:14,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:14,184 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:14,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:14,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:14,285 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:14,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:14,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:14,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:14,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,293 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:14,293 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:14,293 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:14,293 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:14,294 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,294 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,295 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:14,295 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:14,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:14,296 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,296 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:14,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,296 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:14,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,297 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:14,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,297 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:14,297 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:14,297 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:14,298 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:14,298 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:14,298 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:14,298 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. started... 2023-03-08 11:54:14,298 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c154d984bcb433ac75837c927dc4662b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:14,315 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035adeee61a27e411886e24319742f6fe1 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601035adeee61a27e411886e24319742f6fe1 2023-03-08 11:54:14,321 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/cde094d5197d4ea3adbf0d7a353c5ae4 2023-03-08 11:54:14,328 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/cde094d5197d4ea3adbf0d7a353c5ae4 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/cde094d5197d4ea3adbf0d7a353c5ae4 2023-03-08 11:54:14,337 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/cde094d5197d4ea3adbf0d7a353c5ae4, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 11:54:14,338 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c154d984bcb433ac75837c927dc4662b in 40ms, sequenceid=33, compaction requested=true 2023-03-08 11:54:14,339 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:14,339 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:14,339 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:14,339 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:14,339 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,339 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:14,339 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:14,346 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:14,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:14,346 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,346 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:14,346 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:14,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:14,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:14,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:14,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67a74e3[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:14,348 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:14,348 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,350 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,350 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:14,350 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,350 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:14,350 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:14,350 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:14,352 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:14,352 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:14,352 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,352 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:14,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:14,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:14,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:14,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:14,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:14,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:14,355 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:14,355 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:14,355 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:14,356 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:14,356 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,356 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:14,357 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,357 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:14,357 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:14,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:14,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:14,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:14,358 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:14,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:14,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,359 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:14,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,363 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:14,363 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:14,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:14,363 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:14,363 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:14,363 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:14,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:14,363 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,363 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,363 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,363 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,363 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,363 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:14,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:14,364 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:14,364 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:14,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:14,464 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:14,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:14,470 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:14,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:14,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,472 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:14,472 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:14,472 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:14,472 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:14,473 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,473 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,474 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,474 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:14,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:14,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:14,474 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:14,474 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:14,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:14,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,475 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,476 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:14,476 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:14,476 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:14,476 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:14,476 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:14,476 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:14,476 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:14,476 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:14,476 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. started... 2023-03-08 11:54:14,476 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c154d984bcb433ac75837c927dc4662b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:14,897 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907f2b70202717b4221849bfd423cf7f302 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907f2b70202717b4221849bfd423cf7f302 2023-03-08 11:54:15,305 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/bd77127477864c0cb0feeb2bfb54b6cf 2023-03-08 11:54:15,312 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/bd77127477864c0cb0feeb2bfb54b6cf as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/bd77127477864c0cb0feeb2bfb54b6cf 2023-03-08 11:54:15,318 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/bd77127477864c0cb0feeb2bfb54b6cf, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 11:54:15,319 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c154d984bcb433ac75837c927dc4662b in 843ms, sequenceid=37, compaction requested=true 2023-03-08 11:54:15,319 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:15,319 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:15,319 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:15,319 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:15,319 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,319 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:15,319 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:15,322 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,323 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,323 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:15,324 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:15,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:15,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f44ab81[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:15,326 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:15,326 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,330 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,330 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:15,330 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:15,330 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,330 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:15,330 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:15,332 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:15,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:15,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:15,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:15,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:15,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:15,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:15,335 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:15,335 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:15,335 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:15,335 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:15,335 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,335 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:15,337 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:15,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:15,337 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,338 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:15,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:15,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:15,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:15,338 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:15,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:15,340 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,343 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:15,343 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,343 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,343 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:15,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:15,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:15,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:15,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:15,343 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:15,343 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,344 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:15,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:15,344 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,344 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:15,344 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:15,344 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,344 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,444 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:15,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:15,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:15,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:15,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,452 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:15,452 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:15,452 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:15,452 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:15,453 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,453 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,454 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:15,454 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:15,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:15,454 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:15,454 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:15,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:15,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:15,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:15,455 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:15,455 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:15,456 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:15,456 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:15,456 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:15,456 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:15,456 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. started... 2023-03-08 11:54:15,456 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c154d984bcb433ac75837c927dc4662b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:15,874 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511207db5657ccac5435a941c50b22643def1 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511207db5657ccac5435a941c50b22643def1 2023-03-08 11:54:16,281 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/343655d1cb464b58a07b4e01bc25cc52 2023-03-08 11:54:16,288 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/343655d1cb464b58a07b4e01bc25cc52 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/343655d1cb464b58a07b4e01bc25cc52 2023-03-08 11:54:16,295 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/343655d1cb464b58a07b4e01bc25cc52, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 11:54:16,297 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c154d984bcb433ac75837c927dc4662b in 841ms, sequenceid=41, compaction requested=true 2023-03-08 11:54:16,297 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:16,297 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:16,297 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:16,297 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:16,297 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,297 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:16,297 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:16,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,299 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:16,299 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,299 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7817ce97[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:16,301 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:16,301 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,303 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,303 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:16,303 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:16,303 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:16,303 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,303 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:16,305 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,305 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:16,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:16,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:16,305 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:16,306 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:16,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:16,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:16,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:16,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:16,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:16,309 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:16,309 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:16,309 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:16,309 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,309 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:16,311 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,311 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:16,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,311 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:16,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:16,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:16,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:16,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:16,312 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:16,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,313 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:16,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,321 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,321 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:16,321 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:16,321 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:16,321 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:16,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 11:54:16,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:16,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:16,322 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:16,321 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,322 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:16,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:16,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:16,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:16,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:16,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,422 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:16,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:16,425 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-08 11:54:16,427 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:54:16,428 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:54:16,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:16,429 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277056429 2023-03-08 11:54:16,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 11:54:16,431 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-08 11:54:16,431 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:16,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x31254ceb to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:54:16,436 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9139c79, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:54:16,436 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:16,498 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ff4738c346a74492813d5f25a222ec81 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ff4738c346a74492813d5f25a222ec81 2023-03-08 11:54:16,499 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:16,519 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:54:16,521 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34456, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:54:16,531 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:54:16,534 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42864, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:54:16,551 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/15be738b71314f49962a8a71d152335d first=Optional[r3] last=Optional[r6] 2023-03-08 11:54:16,552 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/15be738b71314f49962a8a71d152335d}] 2023-03-08 11:54:16,558 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/15be738b71314f49962a8a71d152335d for inclusion in c154d984bcb433ac75837c927dc4662b/family1 2023-03-08 11:54:16,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 11:54:16,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:54:16,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:16,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/15be738b71314f49962a8a71d152335d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rqc1k5vgdccrje2uqocm36brqg756e9cmnl5pvdna4hduiei9kn668hu9n32uo9r/family1/15be738b71314f49962a8a71d152335d 2023-03-08 11:54:16,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rqc1k5vgdccrje2uqocm36brqg756e9cmnl5pvdna4hduiei9kn668hu9n32uo9r/family1/15be738b71314f49962a8a71d152335d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/3797a1e620514052a3185ebe2982764f_SeqId_44_ 2023-03-08 11:54:16,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rqc1k5vgdccrje2uqocm36brqg756e9cmnl5pvdna4hduiei9kn668hu9n32uo9r/family1/15be738b71314f49962a8a71d152335d into c154d984bcb433ac75837c927dc4662b/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/3797a1e620514052a3185ebe2982764f_SeqId_44_ - updating store file list. 2023-03-08 11:54:16,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/3797a1e620514052a3185ebe2982764f_SeqId_44_ into c154d984bcb433ac75837c927dc4662b/family1 2023-03-08 11:54:16,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rqc1k5vgdccrje2uqocm36brqg756e9cmnl5pvdna4hduiei9kn668hu9n32uo9r/family1/15be738b71314f49962a8a71d152335d into c154d984bcb433ac75837c927dc4662b/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/3797a1e620514052a3185ebe2982764f_SeqId_44_) 2023-03-08 11:54:16,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rqc1k5vgdccrje2uqocm36brqg756e9cmnl5pvdna4hduiei9kn668hu9n32uo9r/family1/15be738b71314f49962a8a71d152335d 2023-03-08 11:54:16,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 11:54:16,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 11:54:16,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): c154d984bcb433ac75837c927dc4662b/family1 is initiating minor compaction 2023-03-08 11:54:16,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@d8f8943; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:16,577 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c154d984bcb433ac75837c927dc4662b/family1 in testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:54:16,577 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/4cc5b7cc14c946f1a71764a5aecbccd5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/a1e07ad8e735439684d333a32628f2c9, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/19c45b36a9fc46378acf169423e7d6a8, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/b24732e0f01b4f8c8b32ecec58177445, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/90a267d0035b42869ec6c0bccd4c0272, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/76fc979161c844f5a5dc838b425f754e, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/cde094d5197d4ea3adbf0d7a353c5ae4, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/bd77127477864c0cb0feeb2bfb54b6cf, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/343655d1cb464b58a07b4e01bc25cc52, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/3797a1e620514052a3185ebe2982764f_SeqId_44_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp, totalSize=50.0 K 2023-03-08 11:54:16,577 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 4cc5b7cc14c946f1a71764a5aecbccd5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 11:54:16,577 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting a1e07ad8e735439684d333a32628f2c9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 11:54:16,578 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 19c45b36a9fc46378acf169423e7d6a8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 11:54:16,578 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting b24732e0f01b4f8c8b32ecec58177445, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 11:54:16,578 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 90a267d0035b42869ec6c0bccd4c0272, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 11:54:16,578 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 76fc979161c844f5a5dc838b425f754e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 11:54:16,578 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting cde094d5197d4ea3adbf0d7a353c5ae4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 11:54:16,578 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting bd77127477864c0cb0feeb2bfb54b6cf, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 11:54:16,578 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 343655d1cb464b58a07b4e01bc25cc52, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 11:54:16,578 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 3797a1e620514052a3185ebe2982764f_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 11:54:16,580 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:16,581 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:16,583 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228c8e6ce38f2c149ef8bfbe59f7eff5e5f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228c8e6ce38f2c149ef8bfbe59f7eff5e5f 2023-03-08 11:54:16,585 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512311d89783d170c41b49d26b73c0c32cbb0 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512311d89783d170c41b49d26b73c0c32cbb0 2023-03-08 11:54:16,587 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101d9ebb3767c254870b37870a687af2d30 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101d9ebb3767c254870b37870a687af2d30 2023-03-08 11:54:16,588 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601035adeee61a27e411886e24319742f6fe1 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601035adeee61a27e411886e24319742f6fe1 2023-03-08 11:54:16,588 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-08 11:54:16,588 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 11:54:16,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:16,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:16,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:16,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:16,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:16,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:16,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:16,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:16,621 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): c154d984bcb433ac75837c927dc4662b#family1#compaction#34 average throughput is 0.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:54:17,038 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e523e607234c4b2fb11ad38b26edf94d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205e523e607234c4b2fb11ad38b26edf94d 2023-03-08 11:54:17,040 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:17,066 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e9b04cd324734ba99444f79e77208e91 first=Optional[r1] last=Optional[r2] 2023-03-08 11:54:17,067 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e9b04cd324734ba99444f79e77208e91}] 2023-03-08 11:54:17,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e9b04cd324734ba99444f79e77208e91 for inclusion in c154d984bcb433ac75837c927dc4662b/family1 2023-03-08 11:54:17,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-08 11:54:17,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:54:17,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:17,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e9b04cd324734ba99444f79e77208e91 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__hp9gjfcpatd75jsl0sskmmmsvgg3au10s4rmnhra3hqelgm7l2e5du21kupmuv3l/family1/e9b04cd324734ba99444f79e77208e91 2023-03-08 11:54:17,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__hp9gjfcpatd75jsl0sskmmmsvgg3au10s4rmnhra3hqelgm7l2e5du21kupmuv3l/family1/e9b04cd324734ba99444f79e77208e91 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/f77c740fc0e94592bc483d0d68a41378_SeqId_46_ 2023-03-08 11:54:17,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__hp9gjfcpatd75jsl0sskmmmsvgg3au10s4rmnhra3hqelgm7l2e5du21kupmuv3l/family1/e9b04cd324734ba99444f79e77208e91 into c154d984bcb433ac75837c927dc4662b/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/f77c740fc0e94592bc483d0d68a41378_SeqId_46_ - updating store file list. 2023-03-08 11:54:17,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/f77c740fc0e94592bc483d0d68a41378_SeqId_46_ into c154d984bcb433ac75837c927dc4662b/family1 2023-03-08 11:54:17,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__hp9gjfcpatd75jsl0sskmmmsvgg3au10s4rmnhra3hqelgm7l2e5du21kupmuv3l/family1/e9b04cd324734ba99444f79e77208e91 into c154d984bcb433ac75837c927dc4662b/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/f77c740fc0e94592bc483d0d68a41378_SeqId_46_) 2023-03-08 11:54:17,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__hp9gjfcpatd75jsl0sskmmmsvgg3au10s4rmnhra3hqelgm7l2e5du21kupmuv3l/family1/e9b04cd324734ba99444f79e77208e91 2023-03-08 11:54:17,085 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:17,086 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:17,087 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120178a68c41550e475f8b4046caebe4fcc1 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120178a68c41550e475f8b4046caebe4fcc1 2023-03-08 11:54:17,088 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205a156faab4fbb47a092998b0f1c3e47dd to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205a156faab4fbb47a092998b0f1c3e47dd 2023-03-08 11:54:17,088 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-08 11:54:17,088 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 11:54:17,089 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:17,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:17,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:17,090 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:54:17,091 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 664 msec 2023-03-08 11:54:17,212 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=260 (was 254) - Thread LEAK? -, OpenFileDescriptor=631 (was 608) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=215 (was 198) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=7474 (was 8074) 2023-03-08 11:54:17,223 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=260, OpenFileDescriptor=631, MaxFileDescriptor=60000, SystemLoadAverage=215, ProcessCount=170, AvailableMemoryMB=7474 2023-03-08 11:54:17,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 11:54:17,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:17,229 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:54:17,229 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-08 11:54:17,230 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:54:17,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 11:54:17,232 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:54:17,232 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:54:17,232 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:54:17,233 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950 empty. 2023-03-08 11:54:17,233 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e empty. 2023-03-08 11:54:17,233 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50 empty. 2023-03-08 11:54:17,233 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:54:17,233 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:54:17,233 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:54:17,234 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-08 11:54:17,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 11:54:17,445 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/.tmp/family1/b6eb027a67524778b5a9d48c7499ce3a as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/b6eb027a67524778b5a9d48c7499ce3a 2023-03-08 11:54:17,450 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in c154d984bcb433ac75837c927dc4662b/family1 of c154d984bcb433ac75837c927dc4662b into b6eb027a67524778b5a9d48c7499ce3a(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:54:17,450 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:54:17,451 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678276456577; duration=0sec 2023-03-08 11:54:17,451 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:17,467 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/4cc5b7cc14c946f1a71764a5aecbccd5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/a1e07ad8e735439684d333a32628f2c9, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/19c45b36a9fc46378acf169423e7d6a8, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/b24732e0f01b4f8c8b32ecec58177445, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/90a267d0035b42869ec6c0bccd4c0272, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/76fc979161c844f5a5dc838b425f754e, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/cde094d5197d4ea3adbf0d7a353c5ae4, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/bd77127477864c0cb0feeb2bfb54b6cf, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/343655d1cb464b58a07b4e01bc25cc52, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/3797a1e620514052a3185ebe2982764f_SeqId_44_] to archive 2023-03-08 11:54:17,468 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:17,470 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/4cc5b7cc14c946f1a71764a5aecbccd5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/4cc5b7cc14c946f1a71764a5aecbccd5 2023-03-08 11:54:17,471 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/a1e07ad8e735439684d333a32628f2c9 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/a1e07ad8e735439684d333a32628f2c9 2023-03-08 11:54:17,473 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/19c45b36a9fc46378acf169423e7d6a8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/19c45b36a9fc46378acf169423e7d6a8 2023-03-08 11:54:17,474 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/b24732e0f01b4f8c8b32ecec58177445 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/b24732e0f01b4f8c8b32ecec58177445 2023-03-08 11:54:17,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/90a267d0035b42869ec6c0bccd4c0272 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/90a267d0035b42869ec6c0bccd4c0272 2023-03-08 11:54:17,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/76fc979161c844f5a5dc838b425f754e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/76fc979161c844f5a5dc838b425f754e 2023-03-08 11:54:17,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/cde094d5197d4ea3adbf0d7a353c5ae4 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/cde094d5197d4ea3adbf0d7a353c5ae4 2023-03-08 11:54:17,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/bd77127477864c0cb0feeb2bfb54b6cf to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/bd77127477864c0cb0feeb2bfb54b6cf 2023-03-08 11:54:17,481 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/343655d1cb464b58a07b4e01bc25cc52 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/343655d1cb464b58a07b4e01bc25cc52 2023-03-08 11:54:17,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/3797a1e620514052a3185ebe2982764f_SeqId_44_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/family1/3797a1e620514052a3185ebe2982764f_SeqId_44_ 2023-03-08 11:54:17,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 11:54:17,642 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-08 11:54:17,644 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 91d8ee5ab3b818ebe215b0c4873b5950, NAME => 'testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:54:17,644 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => 8fd6f903c75ee810ecad794fff5abb6e, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:54:17,644 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 8bbbd6b703fb6a51c3770af58f047d50, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:54:17,673 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:54:17,674 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x31254ceb to 127.0.0.1:51863 2023-03-08 11:54:17,674 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:54:17,675 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:17,675 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing 8fd6f903c75ee810ecad794fff5abb6e, disabling compactions & flushes 2023-03-08 11:54:17,675 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:17,675 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:17,675 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. after waiting 0 ms 2023-03-08 11:54:17,675 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:54:17,675 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:17,676 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x02f847fd to 127.0.0.1:51863 2023-03-08 11:54:17,676 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:54:17,676 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:54:17,676 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:17,676 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x127ee2f7 to 127.0.0.1:51863 2023-03-08 11:54:17,676 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for 8fd6f903c75ee810ecad794fff5abb6e: 2023-03-08 11:54:17,676 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:54:17,676 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:54:17,677 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4e76a679 to 127.0.0.1:51863 2023-03-08 11:54:17,677 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:54:17,680 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:17,680 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 8bbbd6b703fb6a51c3770af58f047d50, disabling compactions & flushes 2023-03-08 11:54:17,680 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:17,680 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:17,680 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. after waiting 0 ms 2023-03-08 11:54:17,680 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:17,680 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:17,680 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 8bbbd6b703fb6a51c3770af58f047d50: 2023-03-08 11:54:17,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 11:54:18,076 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:18,076 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 91d8ee5ab3b818ebe215b0c4873b5950, disabling compactions & flushes 2023-03-08 11:54:18,076 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:18,076 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:18,076 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. after waiting 0 ms 2023-03-08 11:54:18,076 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:18,076 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:18,076 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 91d8ee5ab3b818ebe215b0c4873b5950: 2023-03-08 11:54:18,079 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:54:18,080 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678276458080"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276458080"}]},"ts":"1678276458080"} 2023-03-08 11:54:18,080 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678276458080"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276458080"}]},"ts":"1678276458080"} 2023-03-08 11:54:18,080 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678276458080"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276458080"}]},"ts":"1678276458080"} 2023-03-08 11:54:18,083 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 11:54:18,084 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:54:18,084 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276458084"}]},"ts":"1678276458084"} 2023-03-08 11:54:18,085 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-08 11:54:18,089 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=91d8ee5ab3b818ebe215b0c4873b5950, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8bbbd6b703fb6a51c3770af58f047d50, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8fd6f903c75ee810ecad794fff5abb6e, ASSIGN}] 2023-03-08 11:54:18,091 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8fd6f903c75ee810ecad794fff5abb6e, ASSIGN 2023-03-08 11:54:18,091 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8bbbd6b703fb6a51c3770af58f047d50, ASSIGN 2023-03-08 11:54:18,091 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=91d8ee5ab3b818ebe215b0c4873b5950, ASSIGN 2023-03-08 11:54:18,091 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8fd6f903c75ee810ecad794fff5abb6e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:54:18,092 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8bbbd6b703fb6a51c3770af58f047d50, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:54:18,092 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=91d8ee5ab3b818ebe215b0c4873b5950, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:54:18,244 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=91d8ee5ab3b818ebe215b0c4873b5950, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:18,244 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=8bbbd6b703fb6a51c3770af58f047d50, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:18,244 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=8fd6f903c75ee810ecad794fff5abb6e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:18,244 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678276458243"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276458243"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276458243"}]},"ts":"1678276458243"} 2023-03-08 11:54:18,244 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678276458243"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276458243"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276458243"}]},"ts":"1678276458243"} 2023-03-08 11:54:18,244 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678276458243"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276458243"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276458243"}]},"ts":"1678276458243"} 2023-03-08 11:54:18,246 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure 91d8ee5ab3b818ebe215b0c4873b5950, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:18,246 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure 8fd6f903c75ee810ecad794fff5abb6e, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:18,247 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 8bbbd6b703fb6a51c3770af58f047d50, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:18,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 11:54:18,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:18,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8fd6f903c75ee810ecad794fff5abb6e, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 11:54:18,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:54:18,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:18,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:54:18,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:54:18,403 INFO [StoreOpener-8fd6f903c75ee810ecad794fff5abb6e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:54:18,404 DEBUG [StoreOpener-8fd6f903c75ee810ecad794fff5abb6e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1 2023-03-08 11:54:18,405 DEBUG [StoreOpener-8fd6f903c75ee810ecad794fff5abb6e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1 2023-03-08 11:54:18,405 INFO [StoreOpener-8fd6f903c75ee810ecad794fff5abb6e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8fd6f903c75ee810ecad794fff5abb6e columnFamilyName family1 2023-03-08 11:54:18,407 INFO [StoreOpener-8fd6f903c75ee810ecad794fff5abb6e-1] regionserver.HStore(310): Store=8fd6f903c75ee810ecad794fff5abb6e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:18,407 INFO [StoreOpener-8fd6f903c75ee810ecad794fff5abb6e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:54:18,409 DEBUG [StoreOpener-8fd6f903c75ee810ecad794fff5abb6e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family2 2023-03-08 11:54:18,409 DEBUG [StoreOpener-8fd6f903c75ee810ecad794fff5abb6e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family2 2023-03-08 11:54:18,409 INFO [StoreOpener-8fd6f903c75ee810ecad794fff5abb6e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8fd6f903c75ee810ecad794fff5abb6e columnFamilyName family2 2023-03-08 11:54:18,411 INFO [StoreOpener-8fd6f903c75ee810ecad794fff5abb6e-1] regionserver.HStore(310): Store=8fd6f903c75ee810ecad794fff5abb6e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:18,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:54:18,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:54:18,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:54:18,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:54:18,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:54:18,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8fd6f903c75ee810ecad794fff5abb6e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74118169, jitterRate=0.10444678366184235}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:54:18,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8fd6f903c75ee810ecad794fff5abb6e: 2023-03-08 11:54:18,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e., pid=32, masterSystemTime=1678276458398 2023-03-08 11:54:18,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:18,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:18,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:18,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 91d8ee5ab3b818ebe215b0c4873b5950, NAME => 'testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 11:54:18,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:54:18,421 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=8fd6f903c75ee810ecad794fff5abb6e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:18,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:18,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:54:18,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:54:18,421 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678276458421"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276458421"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276458421"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276458421"}]},"ts":"1678276458421"} 2023-03-08 11:54:18,422 INFO [StoreOpener-91d8ee5ab3b818ebe215b0c4873b5950-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:54:18,424 DEBUG [StoreOpener-91d8ee5ab3b818ebe215b0c4873b5950-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950/family1 2023-03-08 11:54:18,424 DEBUG [StoreOpener-91d8ee5ab3b818ebe215b0c4873b5950-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950/family1 2023-03-08 11:54:18,424 INFO [StoreOpener-91d8ee5ab3b818ebe215b0c4873b5950-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 91d8ee5ab3b818ebe215b0c4873b5950 columnFamilyName family1 2023-03-08 11:54:18,425 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=30 2023-03-08 11:54:18,425 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure 8fd6f903c75ee810ecad794fff5abb6e, server=jenkins-hbase4.apache.org,45551,1678276414946 in 177 msec 2023-03-08 11:54:18,425 INFO [StoreOpener-91d8ee5ab3b818ebe215b0c4873b5950-1] regionserver.HStore(310): Store=91d8ee5ab3b818ebe215b0c4873b5950/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:18,426 INFO [StoreOpener-91d8ee5ab3b818ebe215b0c4873b5950-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:54:18,426 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8fd6f903c75ee810ecad794fff5abb6e, ASSIGN in 336 msec 2023-03-08 11:54:18,427 DEBUG [StoreOpener-91d8ee5ab3b818ebe215b0c4873b5950-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950/family2 2023-03-08 11:54:18,427 DEBUG [StoreOpener-91d8ee5ab3b818ebe215b0c4873b5950-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950/family2 2023-03-08 11:54:18,428 INFO [StoreOpener-91d8ee5ab3b818ebe215b0c4873b5950-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 91d8ee5ab3b818ebe215b0c4873b5950 columnFamilyName family2 2023-03-08 11:54:18,429 INFO [StoreOpener-91d8ee5ab3b818ebe215b0c4873b5950-1] regionserver.HStore(310): Store=91d8ee5ab3b818ebe215b0c4873b5950/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:18,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:54:18,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:54:18,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:54:18,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:54:18,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:54:18,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 91d8ee5ab3b818ebe215b0c4873b5950; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59217663, jitterRate=-0.11758805811405182}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:54:18,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 91d8ee5ab3b818ebe215b0c4873b5950: 2023-03-08 11:54:18,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950., pid=31, masterSystemTime=1678276458398 2023-03-08 11:54:18,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:18,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:18,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:18,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8bbbd6b703fb6a51c3770af58f047d50, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 11:54:18,437 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=91d8ee5ab3b818ebe215b0c4873b5950, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:18,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:54:18,437 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678276458437"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276458437"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276458437"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276458437"}]},"ts":"1678276458437"} 2023-03-08 11:54:18,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:18,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:54:18,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:54:18,439 INFO [StoreOpener-8bbbd6b703fb6a51c3770af58f047d50-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:54:18,441 DEBUG [StoreOpener-8bbbd6b703fb6a51c3770af58f047d50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50/family1 2023-03-08 11:54:18,441 DEBUG [StoreOpener-8bbbd6b703fb6a51c3770af58f047d50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50/family1 2023-03-08 11:54:18,441 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-03-08 11:54:18,441 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure 91d8ee5ab3b818ebe215b0c4873b5950, server=jenkins-hbase4.apache.org,45551,1678276414946 in 193 msec 2023-03-08 11:54:18,441 INFO [StoreOpener-8bbbd6b703fb6a51c3770af58f047d50-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8bbbd6b703fb6a51c3770af58f047d50 columnFamilyName family1 2023-03-08 11:54:18,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=91d8ee5ab3b818ebe215b0c4873b5950, ASSIGN in 352 msec 2023-03-08 11:54:18,442 INFO [StoreOpener-8bbbd6b703fb6a51c3770af58f047d50-1] regionserver.HStore(310): Store=8bbbd6b703fb6a51c3770af58f047d50/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:18,443 INFO [StoreOpener-8bbbd6b703fb6a51c3770af58f047d50-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:54:18,444 DEBUG [StoreOpener-8bbbd6b703fb6a51c3770af58f047d50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50/family2 2023-03-08 11:54:18,444 DEBUG [StoreOpener-8bbbd6b703fb6a51c3770af58f047d50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50/family2 2023-03-08 11:54:18,444 INFO [StoreOpener-8bbbd6b703fb6a51c3770af58f047d50-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8bbbd6b703fb6a51c3770af58f047d50 columnFamilyName family2 2023-03-08 11:54:18,445 INFO [StoreOpener-8bbbd6b703fb6a51c3770af58f047d50-1] regionserver.HStore(310): Store=8bbbd6b703fb6a51c3770af58f047d50/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:18,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:54:18,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:54:18,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:54:18,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:54:18,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:54:18,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8bbbd6b703fb6a51c3770af58f047d50; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62969644, jitterRate=-0.061679184436798096}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:54:18,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8bbbd6b703fb6a51c3770af58f047d50: 2023-03-08 11:54:18,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50., pid=33, masterSystemTime=1678276458398 2023-03-08 11:54:18,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:18,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:18,453 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=8bbbd6b703fb6a51c3770af58f047d50, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:18,453 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678276458453"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276458453"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276458453"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276458453"}]},"ts":"1678276458453"} 2023-03-08 11:54:18,456 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-03-08 11:54:18,456 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 8bbbd6b703fb6a51c3770af58f047d50, server=jenkins-hbase4.apache.org,45551,1678276414946 in 208 msec 2023-03-08 11:54:18,459 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=27 2023-03-08 11:54:18,459 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8bbbd6b703fb6a51c3770af58f047d50, ASSIGN in 367 msec 2023-03-08 11:54:18,459 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:54:18,460 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276458460"}]},"ts":"1678276458460"} 2023-03-08 11:54:18,461 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-08 11:54:18,464 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:54:18,465 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2390 sec 2023-03-08 11:54:18,532 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-08 11:54:19,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 11:54:19,336 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-08 11:54:19,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:19,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:19,345 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-08 11:54:19,345 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:19,345 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-08 11:54:19,345 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-08 11:54:19,346 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:19,346 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:19,347 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:19,347 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:19,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:19,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:19,347 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:19,347 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:19,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:19,348 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:19,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:19,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:19,348 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:19,348 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:19,348 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-08 11:54:19,348 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:19,348 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-08 11:54:19,349 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:19,349 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. started... 2023-03-08 11:54:19,349 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:19,349 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:54:19,349 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8fd6f903c75ee810ecad794fff5abb6e 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-08 11:54:19,349 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:19,349 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:19,349 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. started... 2023-03-08 11:54:19,350 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. started... 2023-03-08 11:54:19,350 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 91d8ee5ab3b818ebe215b0c4873b5950: 2023-03-08 11:54:19,351 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8bbbd6b703fb6a51c3770af58f047d50: 2023-03-08 11:54:19,351 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:19,351 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:19,351 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:54:19,351 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:54:19,766 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303081a24f2bb83554c2bb7a93d93de54b5b2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303081a24f2bb83554c2bb7a93d93de54b5b2 2023-03-08 11:54:20,172 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/.tmp/family1/e22e670e0e60446c95635c583b53f0fc 2023-03-08 11:54:20,177 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/.tmp/family1/e22e670e0e60446c95635c583b53f0fc as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/e22e670e0e60446c95635c583b53f0fc 2023-03-08 11:54:20,183 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/e22e670e0e60446c95635c583b53f0fc, entries=1, sequenceid=5, filesize=4.9 K 2023-03-08 11:54:20,184 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 8fd6f903c75ee810ecad794fff5abb6e in 835ms, sequenceid=5, compaction requested=false 2023-03-08 11:54:20,185 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8fd6f903c75ee810ecad794fff5abb6e: 2023-03-08 11:54:20,185 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:20,185 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:54:20,185 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:54:20,185 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,185 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-08 11:54:20,185 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-08 11:54:20,186 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,186 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:20,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:20,187 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,187 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:20,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:20,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:20,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:20,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-08 11:54:20,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53664607[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:20,188 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-08 11:54:20,188 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,190 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,190 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-08 11:54:20,190 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-08 11:54:20,190 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:20,190 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,190 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:20,193 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-08 11:54:20,193 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,193 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:20,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:20,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:20,193 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-08 11:54:20,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:20,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:20,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:20,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:20,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-08 11:54:20,195 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-08 11:54:20,195 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:20,195 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:20,195 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,195 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:20,197 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,197 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,197 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:20,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:20,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:20,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:20,197 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:20,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:20,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:20,199 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:20,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,203 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:20,203 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:20,204 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:20,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:20,204 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:20,204 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-08 11:54:20,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:20,204 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,204 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,204 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:20,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:20,204 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:20,204 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:20,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-08 11:54:20,204 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,204 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,304 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:20,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:20,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:20,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:20,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,311 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-08 11:54:20,311 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:20,311 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-08 11:54:20,311 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-08 11:54:20,312 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,312 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,313 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:20,313 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:20,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:20,313 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:20,313 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:20,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,314 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:20,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,314 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,314 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-08 11:54:20,314 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:20,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-08 11:54:20,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:20,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-08 11:54:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. started... 2023-03-08 11:54:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. started... 2023-03-08 11:54:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. started... 2023-03-08 11:54:20,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:54:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8bbbd6b703fb6a51c3770af58f047d50: 2023-03-08 11:54:20,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:20,315 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8fd6f903c75ee810ecad794fff5abb6e 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-08 11:54:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 91d8ee5ab3b818ebe215b0c4873b5950: 2023-03-08 11:54:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:54:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:54:20,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:54:20,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:54:20,731 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308ca12be08b0ab47319dddc8a400f6864d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308ca12be08b0ab47319dddc8a400f6864d 2023-03-08 11:54:21,136 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/.tmp/family1/dacf2a8eed9c4c52b088f956bea873d9 2023-03-08 11:54:21,142 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/.tmp/family1/dacf2a8eed9c4c52b088f956bea873d9 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/dacf2a8eed9c4c52b088f956bea873d9 2023-03-08 11:54:21,146 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/dacf2a8eed9c4c52b088f956bea873d9, entries=2, sequenceid=9, filesize=5.0 K 2023-03-08 11:54:21,147 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 8fd6f903c75ee810ecad794fff5abb6e in 832ms, sequenceid=9, compaction requested=false 2023-03-08 11:54:21,147 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8fd6f903c75ee810ecad794fff5abb6e: 2023-03-08 11:54:21,147 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:21,147 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:54:21,147 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:54:21,147 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,147 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-08 11:54:21,147 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-08 11:54:21,149 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,149 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:21,149 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,149 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:21,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:21,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-08 11:54:21,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66efde4[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:21,151 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-08 11:54:21,151 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,153 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,153 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-08 11:54:21,153 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-08 11:54:21,153 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:21,154 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,154 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:21,155 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-08 11:54:21,155 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,155 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:21,155 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-08 11:54:21,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:21,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:21,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:21,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:21,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-08 11:54:21,158 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-08 11:54:21,158 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:21,158 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:21,158 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,158 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:21,160 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,160 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,160 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:21,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:21,160 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:21,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,161 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:21,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,163 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,164 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,164 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:21,164 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:21,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-08 11:54:21,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:21,164 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:21,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-08 11:54:21,164 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,164 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:21,164 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:21,164 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:21,164 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:21,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:21,165 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:21,165 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:21,265 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:21,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:21,272 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-08 11:54:21,273 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 11:54:21,274 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 11:54:21,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:21,275 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277061275 2023-03-08 11:54:21,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:54:21,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-08 11:54:21,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 11:54:21,276 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x0034b74e to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:54:21,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@15511365, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:54:21,284 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-08 11:54:22,114 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308fe4073b5fdda4b9c9afefe3de8170812 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308fe4073b5fdda4b9c9afefe3de8170812 2023-03-08 11:54:22,115 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 11:54:22,129 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:54:22,130 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34466, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:54:22,137 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:54:22,139 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42866, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:54:22,148 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/3c9e61c4a42e42c99b95079300fadd7e first=Optional[k0] last=Optional[k1] 2023-03-08 11:54:22,149 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/3c9e61c4a42e42c99b95079300fadd7e}] 2023-03-08 11:54:22,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/3c9e61c4a42e42c99b95079300fadd7e for inclusion in 8fd6f903c75ee810ecad794fff5abb6e/family1 2023-03-08 11:54:22,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-08 11:54:22,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 11:54:22,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 8fd6f903c75ee810ecad794fff5abb6e: 2023-03-08 11:54:22,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/3c9e61c4a42e42c99b95079300fadd7e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testScannerOnBulkLoadRefHFiles__thuv53mf0p67b7fnlk9v1f8c58aq6aurq0393mj1d7vg0dedeo5f4pp3rtfs4aub/family1/3c9e61c4a42e42c99b95079300fadd7e 2023-03-08 11:54:22,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testScannerOnBulkLoadRefHFiles__thuv53mf0p67b7fnlk9v1f8c58aq6aurq0393mj1d7vg0dedeo5f4pp3rtfs4aub/family1/3c9e61c4a42e42c99b95079300fadd7e as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/422127b44d8f4d778b969397ab144fdf_SeqId_12_ 2023-03-08 11:54:22,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testScannerOnBulkLoadRefHFiles__thuv53mf0p67b7fnlk9v1f8c58aq6aurq0393mj1d7vg0dedeo5f4pp3rtfs4aub/family1/3c9e61c4a42e42c99b95079300fadd7e into 8fd6f903c75ee810ecad794fff5abb6e/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/422127b44d8f4d778b969397ab144fdf_SeqId_12_ - updating store file list. 2023-03-08 11:54:22,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/422127b44d8f4d778b969397ab144fdf_SeqId_12_ into 8fd6f903c75ee810ecad794fff5abb6e/family1 2023-03-08 11:54:22,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testScannerOnBulkLoadRefHFiles__thuv53mf0p67b7fnlk9v1f8c58aq6aurq0393mj1d7vg0dedeo5f4pp3rtfs4aub/family1/3c9e61c4a42e42c99b95079300fadd7e into 8fd6f903c75ee810ecad794fff5abb6e/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/422127b44d8f4d778b969397ab144fdf_SeqId_12_) 2023-03-08 11:54:22,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testScannerOnBulkLoadRefHFiles__thuv53mf0p67b7fnlk9v1f8c58aq6aurq0393mj1d7vg0dedeo5f4pp3rtfs4aub/family1/3c9e61c4a42e42c99b95079300fadd7e 2023-03-08 11:54:22,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 11:54:22,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 11:54:22,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 8fd6f903c75ee810ecad794fff5abb6e/family1 is initiating minor compaction (all files) 2023-03-08 11:54:22,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@59457120; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:22,165 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8fd6f903c75ee810ecad794fff5abb6e/family1 in testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:54:22,166 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/e22e670e0e60446c95635c583b53f0fc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/dacf2a8eed9c4c52b088f956bea873d9, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/422127b44d8f4d778b969397ab144fdf_SeqId_12_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/.tmp, totalSize=15.1 K 2023-03-08 11:54:22,166 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting e22e670e0e60446c95635c583b53f0fc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276459337 2023-03-08 11:54:22,167 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting dacf2a8eed9c4c52b088f956bea873d9, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276459337 2023-03-08 11:54:22,167 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 422127b44d8f4d778b969397ab144fdf_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276459337 2023-03-08 11:54:22,167 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 11:54:22,168 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:22,170 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303081a24f2bb83554c2bb7a93d93de54b5b2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303081a24f2bb83554c2bb7a93d93de54b5b2 2023-03-08 11:54:22,171 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308ca12be08b0ab47319dddc8a400f6864d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308ca12be08b0ab47319dddc8a400f6864d 2023-03-08 11:54:22,171 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-08 11:54:22,171 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 11:54:22,172 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-08 11:54:22,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 11:54:22,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:22,173 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 11:54:22,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 900 msec 2023-03-08 11:54:22,182 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8fd6f903c75ee810ecad794fff5abb6e#family1#compaction#37 average throughput is 0.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:54:22,209 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=244 (was 260), OpenFileDescriptor=587 (was 631), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=204 (was 215), ProcessCount=170 (was 170), AvailableMemoryMB=7488 (was 7474) - AvailableMemoryMB LEAK? - 2023-03-08 11:54:22,223 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=244, OpenFileDescriptor=587, MaxFileDescriptor=60000, SystemLoadAverage=204, ProcessCount=170, AvailableMemoryMB=7488 2023-03-08 11:54:22,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 11:54:22,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:22,229 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:54:22,229 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-08 11:54:22,229 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:54:22,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 11:54:22,231 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9 2023-03-08 11:54:22,231 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9 empty. 2023-03-08 11:54:22,232 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9 2023-03-08 11:54:22,232 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-08 11:54:22,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 11:54:22,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 11:54:22,641 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 11:54:22,643 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6334f697e0e777acd41789a8c25bc8c9, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:54:22,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 11:54:23,050 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:23,050 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 6334f697e0e777acd41789a8c25bc8c9, disabling compactions & flushes 2023-03-08 11:54:23,050 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:23,050 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:23,050 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. after waiting 0 ms 2023-03-08 11:54:23,050 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:23,050 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:23,050 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:23,053 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:54:23,054 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678276463054"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276463054"}]},"ts":"1678276463054"} 2023-03-08 11:54:23,055 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 11:54:23,056 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:54:23,057 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276463056"}]},"ts":"1678276463056"} 2023-03-08 11:54:23,058 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-08 11:54:23,060 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=6334f697e0e777acd41789a8c25bc8c9, ASSIGN}] 2023-03-08 11:54:23,062 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=6334f697e0e777acd41789a8c25bc8c9, ASSIGN 2023-03-08 11:54:23,063 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=6334f697e0e777acd41789a8c25bc8c9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:54:23,214 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=6334f697e0e777acd41789a8c25bc8c9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:23,214 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678276463214"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276463214"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276463214"}]},"ts":"1678276463214"} 2023-03-08 11:54:23,215 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 6334f697e0e777acd41789a8c25bc8c9, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:23,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 11:54:23,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:23,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6334f697e0e777acd41789a8c25bc8c9, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:54:23,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 6334f697e0e777acd41789a8c25bc8c9 2023-03-08 11:54:23,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:23,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6334f697e0e777acd41789a8c25bc8c9 2023-03-08 11:54:23,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6334f697e0e777acd41789a8c25bc8c9 2023-03-08 11:54:23,372 INFO [StoreOpener-6334f697e0e777acd41789a8c25bc8c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6334f697e0e777acd41789a8c25bc8c9 2023-03-08 11:54:23,373 DEBUG [StoreOpener-6334f697e0e777acd41789a8c25bc8c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1 2023-03-08 11:54:23,373 DEBUG [StoreOpener-6334f697e0e777acd41789a8c25bc8c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1 2023-03-08 11:54:23,374 INFO [StoreOpener-6334f697e0e777acd41789a8c25bc8c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6334f697e0e777acd41789a8c25bc8c9 columnFamilyName family1 2023-03-08 11:54:23,375 INFO [StoreOpener-6334f697e0e777acd41789a8c25bc8c9-1] regionserver.HStore(310): Store=6334f697e0e777acd41789a8c25bc8c9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:23,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9 2023-03-08 11:54:23,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9 2023-03-08 11:54:23,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6334f697e0e777acd41789a8c25bc8c9 2023-03-08 11:54:23,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:54:23,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6334f697e0e777acd41789a8c25bc8c9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66188145, jitterRate=-0.01371978223323822}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:54:23,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:23,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9., pid=37, masterSystemTime=1678276463367 2023-03-08 11:54:23,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:23,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:23,384 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=6334f697e0e777acd41789a8c25bc8c9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:23,384 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678276463384"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276463384"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276463384"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276463384"}]},"ts":"1678276463384"} 2023-03-08 11:54:23,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-08 11:54:23,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 6334f697e0e777acd41789a8c25bc8c9, server=jenkins-hbase4.apache.org,45551,1678276414946 in 171 msec 2023-03-08 11:54:23,388 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-08 11:54:23,388 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=6334f697e0e777acd41789a8c25bc8c9, ASSIGN in 327 msec 2023-03-08 11:54:23,389 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:54:23,389 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276463389"}]},"ts":"1678276463389"} 2023-03-08 11:54:23,390 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-08 11:54:23,392 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:54:23,394 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1660 sec 2023-03-08 11:54:23,402 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/.tmp/family1/4d719e88946245d5b1dfdb79ce7f8b5e as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/4d719e88946245d5b1dfdb79ce7f8b5e 2023-03-08 11:54:23,407 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 8fd6f903c75ee810ecad794fff5abb6e/family1 of 8fd6f903c75ee810ecad794fff5abb6e into 4d719e88946245d5b1dfdb79ce7f8b5e(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:54:23,407 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8fd6f903c75ee810ecad794fff5abb6e: 2023-03-08 11:54:23,407 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1678276462165; duration=1sec 2023-03-08 11:54:23,407 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:23,466 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/e22e670e0e60446c95635c583b53f0fc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/dacf2a8eed9c4c52b088f956bea873d9, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/422127b44d8f4d778b969397ab144fdf_SeqId_12_] to archive 2023-03-08 11:54:23,468 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:23,470 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/e22e670e0e60446c95635c583b53f0fc to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/e22e670e0e60446c95635c583b53f0fc 2023-03-08 11:54:23,471 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/dacf2a8eed9c4c52b088f956bea873d9 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/dacf2a8eed9c4c52b088f956bea873d9 2023-03-08 11:54:23,473 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/422127b44d8f4d778b969397ab144fdf_SeqId_12_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/family1/422127b44d8f4d778b969397ab144fdf_SeqId_12_ 2023-03-08 11:54:23,928 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:54:24,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 11:54:24,336 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-08 11:54:24,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:24,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:24,359 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:24,359 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:24,359 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:24,359 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:24,359 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:24,360 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:24,366 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:24,366 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:24,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:24,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:24,366 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:24,366 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:24,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:24,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:24,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:24,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:24,367 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:24,367 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:24,368 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:24,368 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:24,368 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:24,368 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:24,368 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:24,368 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:24,368 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. started... 2023-03-08 11:54:24,368 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6334f697e0e777acd41789a8c25bc8c9 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 11:54:24,533 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:24,791 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113008b4f56bc7bc47618c955afb68d2bcf3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113008b4f56bc7bc47618c955afb68d2bcf3 2023-03-08 11:54:25,195 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/f2960f5204b54902b04206814ab2887d 2023-03-08 11:54:25,200 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/f2960f5204b54902b04206814ab2887d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/f2960f5204b54902b04206814ab2887d 2023-03-08 11:54:25,205 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/f2960f5204b54902b04206814ab2887d, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-08 11:54:25,206 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 6334f697e0e777acd41789a8c25bc8c9 in 838ms, sequenceid=5, compaction requested=false 2023-03-08 11:54:25,206 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:25,207 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:25,207 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:25,207 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:25,207 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,207 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:25,207 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:25,208 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:25,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:25,208 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:25,209 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,209 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:25,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:25,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ccc560b[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:25,210 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:25,210 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,211 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,211 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:25,211 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,211 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:25,211 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:25,211 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:25,212 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,213 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:25,213 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:25,213 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:25,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:25,215 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:25,215 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:25,215 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:25,215 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,215 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:25,217 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,217 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:25,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:25,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,217 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:25,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:25,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:25,217 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:25,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:25,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,218 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,220 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,220 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,220 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,220 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,220 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:25,220 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:25,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:25,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:25,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:25,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:25,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:25,220 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,221 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:25,221 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,221 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,221 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:25,221 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:25,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:25,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:25,321 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:25,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:25,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:25,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:25,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,328 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:25,328 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:25,328 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:25,328 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:25,329 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,329 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,330 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:25,330 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:25,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:25,330 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:25,330 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:25,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:25,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,331 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,331 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:25,331 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:25,331 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:25,331 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:25,331 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:25,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:25,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:25,332 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:25,332 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. started... 2023-03-08 11:54:25,332 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6334f697e0e777acd41789a8c25bc8c9 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 11:54:25,745 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280286adafcebe42aab37604424669c6c2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511280286adafcebe42aab37604424669c6c2 2023-03-08 11:54:26,152 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/c5e661044dae4fdcab213fe0c0041d3d 2023-03-08 11:54:26,157 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/c5e661044dae4fdcab213fe0c0041d3d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/c5e661044dae4fdcab213fe0c0041d3d 2023-03-08 11:54:26,161 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/c5e661044dae4fdcab213fe0c0041d3d, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 11:54:26,162 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 6334f697e0e777acd41789a8c25bc8c9 in 830ms, sequenceid=9, compaction requested=false 2023-03-08 11:54:26,162 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:26,162 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:26,164 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,164 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:26,164 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,164 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:26,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:26,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ae94473[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:26,165 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:26,165 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,170 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,170 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:26,170 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:26,170 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:26,170 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,170 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:26,171 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,171 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:26,171 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:26,171 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:26,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:26,174 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:26,174 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:26,174 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:26,174 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,174 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:26,175 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,175 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,175 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:26,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:26,180 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:26,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,181 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:26,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,187 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,187 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:26,187 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,187 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:26,187 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,187 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:26,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:26,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:26,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:26,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:26,187 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,188 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,188 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,188 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:26,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:26,188 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:26,188 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:26,288 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:26,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:26,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:26,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:26,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,295 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:26,295 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:26,295 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:26,295 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:26,295 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,295 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,297 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,297 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:26,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:26,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:26,297 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:26,297 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:26,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:26,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:26,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:26,298 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:26,298 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:26,298 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:26,298 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:26,298 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:26,298 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:26,299 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. started... 2023-03-08 11:54:26,299 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6334f697e0e777acd41789a8c25bc8c9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:26,712 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120118bd201aaa0d4f3abd3f2c97c2cbbb5a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120118bd201aaa0d4f3abd3f2c97c2cbbb5a 2023-03-08 11:54:27,118 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/4d22b4fc0bb04284af937e39fcd7b5e0 2023-03-08 11:54:27,123 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/4d22b4fc0bb04284af937e39fcd7b5e0 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/4d22b4fc0bb04284af937e39fcd7b5e0 2023-03-08 11:54:27,127 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/4d22b4fc0bb04284af937e39fcd7b5e0, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 11:54:27,128 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6334f697e0e777acd41789a8c25bc8c9 in 829ms, sequenceid=13, compaction requested=true 2023-03-08 11:54:27,128 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:27,128 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:27,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:27,130 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,130 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1bde6357[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:27,132 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:27,132 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,133 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,133 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:27,133 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:27,133 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:27,133 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,134 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:27,136 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,136 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:27,136 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:27,136 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:27,138 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:27,138 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:27,138 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:27,138 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,138 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:27,139 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,139 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:27,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,140 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:27,140 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,141 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,143 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,143 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:27,143 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:27,143 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:27,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:27,143 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:27,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:27,143 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:27,143 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:27,143 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,144 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:27,144 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:27,144 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,144 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,244 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:27,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:27,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:27,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:27,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,250 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:27,250 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:27,250 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:27,250 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:27,251 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,251 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,252 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,252 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:27,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:27,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:27,252 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:27,252 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:27,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:27,253 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:27,253 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:27,253 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:27,253 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:27,253 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:27,254 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:27,254 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. started... 2023-03-08 11:54:27,254 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6334f697e0e777acd41789a8c25bc8c9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:27,667 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120529aa82910f4c4c96a9b77e1e963c2d9d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120529aa82910f4c4c96a9b77e1e963c2d9d 2023-03-08 11:54:28,073 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/1ffbfbf70a864d239647cb05baa30e43 2023-03-08 11:54:28,078 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/1ffbfbf70a864d239647cb05baa30e43 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/1ffbfbf70a864d239647cb05baa30e43 2023-03-08 11:54:28,083 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/1ffbfbf70a864d239647cb05baa30e43, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 11:54:28,084 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6334f697e0e777acd41789a8c25bc8c9 in 829ms, sequenceid=17, compaction requested=true 2023-03-08 11:54:28,084 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:28,084 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:28,084 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:28,084 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:28,084 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,084 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:28,084 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:28,086 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,086 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:28,087 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,087 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:28,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:28,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6aae3c15[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:28,088 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:28,088 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,089 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,089 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:28,089 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,089 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:28,089 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:28,089 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:28,091 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:28,091 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:28,091 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,091 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:28,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:28,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:28,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:28,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:28,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:28,093 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:28,093 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:28,093 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:28,093 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,093 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:28,095 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,095 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:28,095 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:28,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:28,096 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,097 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,100 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,100 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:28,100 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:28,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:28,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:28,100 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:28,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:28,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:28,100 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:28,100 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,100 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,101 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,101 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,101 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,101 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:28,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:28,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:28,101 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:28,201 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:28,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:28,206 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:28,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:28,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,208 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:28,208 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:28,208 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:28,208 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:28,209 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,209 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,210 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,210 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:28,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:28,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:28,210 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:28,210 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:28,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,211 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:28,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,211 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,211 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:28,211 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:28,211 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:28,211 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:28,211 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:28,212 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:28,212 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:28,212 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:28,212 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. started... 2023-03-08 11:54:28,212 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6334f697e0e777acd41789a8c25bc8c9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:28,625 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512281e239a3d32ac473b8fad569da0102013 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512281e239a3d32ac473b8fad569da0102013 2023-03-08 11:54:28,872 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:54:29,031 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/054b8ec5f9a84775b220e1da377a6ab0 2023-03-08 11:54:29,036 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/054b8ec5f9a84775b220e1da377a6ab0 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/054b8ec5f9a84775b220e1da377a6ab0 2023-03-08 11:54:29,040 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/054b8ec5f9a84775b220e1da377a6ab0, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 11:54:29,040 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6334f697e0e777acd41789a8c25bc8c9 in 828ms, sequenceid=21, compaction requested=true 2023-03-08 11:54:29,041 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:29,041 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:29,041 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:29,041 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:29,041 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,041 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:29,041 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:29,045 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,045 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:29,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:29,045 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,045 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:29,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:29,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:29,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:29,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:29,047 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:29,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@348c5345[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:29,047 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,048 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,048 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:29,048 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:29,048 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:29,048 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,048 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:29,049 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:29,049 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:29,049 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,049 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:29,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:29,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:29,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:29,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:29,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:29,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:29,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:29,051 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:29,051 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:29,052 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:29,052 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,052 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:29,054 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,054 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:29,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:29,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,055 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:29,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:29,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:29,055 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:29,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:29,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,056 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:29,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,058 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,058 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,058 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,058 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,058 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,058 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:29,058 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:29,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:29,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:29,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:29,058 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:29,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:29,058 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:29,059 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,059 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:29,059 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:29,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:29,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:29,159 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:29,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:29,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:29,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:29,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,165 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:29,165 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:29,165 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:29,165 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:29,166 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,166 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,167 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,167 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:29,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:29,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:29,167 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,167 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:29,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:29,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,168 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,168 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:29,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,168 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:29,168 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:29,169 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:29,169 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:29,169 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:29,169 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:29,169 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. started... 2023-03-08 11:54:29,169 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6334f697e0e777acd41789a8c25bc8c9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:29,581 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231f750eb13b10a4729a6aa83a28a350d08 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231f750eb13b10a4729a6aa83a28a350d08 2023-03-08 11:54:29,987 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/834cb34757674c8a9779156b70bc7c36 2023-03-08 11:54:29,992 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/834cb34757674c8a9779156b70bc7c36 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/834cb34757674c8a9779156b70bc7c36 2023-03-08 11:54:29,996 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/834cb34757674c8a9779156b70bc7c36, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 11:54:29,997 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6334f697e0e777acd41789a8c25bc8c9 in 828ms, sequenceid=25, compaction requested=true 2023-03-08 11:54:29,997 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:29,997 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:29,997 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:29,997 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:29,997 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,997 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:29,997 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:29,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,999 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:29,999 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:29,999 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:29,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:30,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:30,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@476537b[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:30,000 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:30,001 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,001 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,002 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:30,002 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:30,002 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:30,002 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,002 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:30,003 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:30,003 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,003 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:30,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:30,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:30,003 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:30,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:30,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:30,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:30,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:30,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:30,005 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:30,005 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:30,005 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:30,005 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,005 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:30,008 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,008 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:30,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:30,008 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:30,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:30,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:30,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:30,009 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:30,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,010 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:30,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,012 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,012 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:30,012 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:30,012 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:30,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:30,012 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:30,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:30,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:30,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:30,012 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,012 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,012 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,012 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,012 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,012 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:30,013 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:30,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:30,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:30,113 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:30,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:30,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:30,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:30,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,120 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:30,120 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:30,120 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:30,120 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:30,121 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,121 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,122 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:30,122 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:30,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:30,122 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,122 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:30,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,123 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:30,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,123 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,123 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:30,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,123 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:30,123 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:30,124 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:30,124 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:30,124 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:30,124 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:30,124 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. started... 2023-03-08 11:54:30,124 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6334f697e0e777acd41789a8c25bc8c9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:30,536 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101f4e9a6cc58a2470d83aebfc613913149 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101f4e9a6cc58a2470d83aebfc613913149 2023-03-08 11:54:30,942 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/9d6a481455244d019ffdaf3299875913 2023-03-08 11:54:30,946 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/9d6a481455244d019ffdaf3299875913 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/9d6a481455244d019ffdaf3299875913 2023-03-08 11:54:30,951 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/9d6a481455244d019ffdaf3299875913, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 11:54:30,952 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6334f697e0e777acd41789a8c25bc8c9 in 828ms, sequenceid=29, compaction requested=true 2023-03-08 11:54:30,952 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:30,952 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:30,952 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:30,952 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:30,952 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,952 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:30,952 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:30,953 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,953 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:30,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:30,954 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,954 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:30,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:30,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:30,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:30,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:30,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c5a5c79[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:30,955 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:30,955 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,957 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,957 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:30,957 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:30,957 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:30,957 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,957 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:30,958 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:30,958 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,958 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:30,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,958 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:30,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:30,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:30,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:30,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:30,960 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:30,960 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:30,960 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:30,961 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,961 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:30,962 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,962 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:30,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,962 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:30,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:30,962 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:30,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,964 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:30,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,970 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,970 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:30,970 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:30,970 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:30,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:30,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:30,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:30,970 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,970 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:30,970 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:30,970 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,970 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,970 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:30,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:30,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:30,970 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:30,971 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:31,071 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:31,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:31,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:31,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:31,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,082 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:31,082 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:31,082 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:31,082 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:31,083 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,083 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,084 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:31,084 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:31,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:31,084 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,084 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:31,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:31,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:31,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,086 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:31,086 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:31,086 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:31,086 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:31,086 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:31,086 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:31,086 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. started... 2023-03-08 11:54:31,086 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6334f697e0e777acd41789a8c25bc8c9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:31,501 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010312acdb423bda47d9bf3f034f52fc3e67 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010312acdb423bda47d9bf3f034f52fc3e67 2023-03-08 11:54:31,907 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/ac945b7508ef4f629d91185e155e1835 2023-03-08 11:54:31,912 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/ac945b7508ef4f629d91185e155e1835 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/ac945b7508ef4f629d91185e155e1835 2023-03-08 11:54:31,916 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/ac945b7508ef4f629d91185e155e1835, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 11:54:31,917 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6334f697e0e777acd41789a8c25bc8c9 in 831ms, sequenceid=33, compaction requested=true 2023-03-08 11:54:31,917 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:31,917 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:31,917 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:31,917 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:31,917 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,917 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:31,917 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:31,919 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,919 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:31,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:31,919 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,919 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:31,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:31,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:31,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:31,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:31,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@778f8b73[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:31,921 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:31,921 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,922 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,922 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:31,922 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:31,922 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:31,922 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,922 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:31,928 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:31,928 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:31,928 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,928 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:31,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:31,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:31,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:31,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:31,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:31,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:31,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:31,931 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:31,931 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:31,931 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:31,931 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,931 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:31,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:31,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:31,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:31,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:31,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:31,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:31,933 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:31,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,934 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:31,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,936 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:31,936 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:31,936 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:31,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:31,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:31,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:31,936 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,937 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,937 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,937 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:31,937 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,937 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,937 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:31,937 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:31,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:31,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:31,937 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:32,037 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:32,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:32,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:32,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:32,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,044 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:32,044 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:32,044 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:32,044 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:32,044 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,045 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,046 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,046 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:32,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:32,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:32,046 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,046 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:32,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,047 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:32,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,047 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,047 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:32,047 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,047 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:32,047 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:32,047 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:32,047 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:32,047 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:32,047 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:32,048 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. started... 2023-03-08 11:54:32,048 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6334f697e0e777acd41789a8c25bc8c9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:32,460 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907707bafd0eca64e3db9f5b4ca52ab7273 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20150907707bafd0eca64e3db9f5b4ca52ab7273 2023-03-08 11:54:32,866 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/8ccdcb5f03334c5a888ed1116d5c1f82 2023-03-08 11:54:32,871 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/8ccdcb5f03334c5a888ed1116d5c1f82 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/8ccdcb5f03334c5a888ed1116d5c1f82 2023-03-08 11:54:32,874 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/8ccdcb5f03334c5a888ed1116d5c1f82, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 11:54:32,875 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6334f697e0e777acd41789a8c25bc8c9 in 827ms, sequenceid=37, compaction requested=true 2023-03-08 11:54:32,875 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:32,875 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:32,876 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:32,876 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:32,876 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,876 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:32,876 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:32,878 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,879 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:32,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:32,879 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,879 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:32,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:32,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:32,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:32,880 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:32,880 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@214ff8bc[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:32,880 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:32,880 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,881 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,881 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:32,881 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,881 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:32,881 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:32,881 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:32,883 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:32,883 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,883 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:32,883 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:32,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:32,885 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:32,885 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:32,885 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:32,885 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:32,885 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,885 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:32,886 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,886 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:32,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:32,886 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:32,886 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:32,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:32,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:32,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:32,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:32,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,890 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,890 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:32,890 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:32,890 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:32,890 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:32,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:32,890 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:32,890 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:32,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:32,890 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,891 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,891 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:32,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:32,891 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:32,891 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:32,991 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:32,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:32,996 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:32,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:32,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,998 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:32,998 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:32,998 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 11:54:32,998 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 11:54:32,999 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:32,999 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,000 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,000 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:33,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:33,000 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,000 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,001 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:33,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,001 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,001 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 11:54:33,001 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,001 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 11:54:33,002 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:33,002 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 11:54:33,002 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:33,002 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:33,002 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:33,002 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. started... 2023-03-08 11:54:33,002 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6334f697e0e777acd41789a8c25bc8c9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:33,415 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511201771409497684c89b4124f4d1c592073 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511201771409497684c89b4124f4d1c592073 2023-03-08 11:54:33,820 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/348e07f2de97412abcd66a597543ddcb 2023-03-08 11:54:33,826 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/348e07f2de97412abcd66a597543ddcb as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/348e07f2de97412abcd66a597543ddcb 2023-03-08 11:54:33,831 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/348e07f2de97412abcd66a597543ddcb, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 11:54:33,832 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6334f697e0e777acd41789a8c25bc8c9 in 830ms, sequenceid=41, compaction requested=true 2023-03-08 11:54:33,832 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:33,832 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:33,832 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:33,832 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:33,832 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,832 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 11:54:33,832 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 11:54:33,834 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,834 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:33,834 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,834 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:33,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:33,835 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 11:54:33,835 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17949a65[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:33,835 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 11:54:33,835 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,836 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,836 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 11:54:33,836 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 11:54:33,836 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,837 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:33,836 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:33,839 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,839 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 11:54:33,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:33,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:33,839 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:33,839 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 11:54:33,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:33,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:33,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:33,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:33,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:54:33,841 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 11:54:33,841 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:33,841 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:33,841 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,841 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:33,842 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,842 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,843 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:33,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:33,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:33,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:33,843 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:33,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:33,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:33,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,844 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:33,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,849 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,849 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:33,849 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:33,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:33,849 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:33,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 11:54:33,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:33,849 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,849 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:33,849 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 11:54:33,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:33,849 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:33,849 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:33,849 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:33,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:33,849 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,850 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,850 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,950 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:33,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:33,952 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-08 11:54:33,953 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:54:33,954 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:54:33,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:33,955 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277073955 2023-03-08 11:54:33,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:54:33,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-08 11:54:33,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:33,958 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x65e0b25c to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:54:33,963 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78375fbe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:54:33,963 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:34,807 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e311a5c8366843e7b81a066b677dd92e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103e311a5c8366843e7b81a066b677dd92e 2023-03-08 11:54:34,808 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:34,821 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:54:34,822 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56876, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:54:34,828 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:54:34,829 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54898, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:54:34,838 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7ac05b904c0749aeab8b326b9cd2ac02 first=Optional[r3] last=Optional[r6] 2023-03-08 11:54:34,838 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7ac05b904c0749aeab8b326b9cd2ac02}] 2023-03-08 11:54:34,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7ac05b904c0749aeab8b326b9cd2ac02 for inclusion in 6334f697e0e777acd41789a8c25bc8c9/family1 2023-03-08 11:54:34,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 11:54:34,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:54:34,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:34,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7ac05b904c0749aeab8b326b9cd2ac02 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__sc1nt5ot65l0it2tph619ctu91uerq4himlroove4f7nt2ruj3mbeu6o73t8e880/family1/7ac05b904c0749aeab8b326b9cd2ac02 2023-03-08 11:54:34,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__sc1nt5ot65l0it2tph619ctu91uerq4himlroove4f7nt2ruj3mbeu6o73t8e880/family1/7ac05b904c0749aeab8b326b9cd2ac02 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/6c35c9ae65184281be840e2a8cc2b5bc_SeqId_44_ 2023-03-08 11:54:34,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__sc1nt5ot65l0it2tph619ctu91uerq4himlroove4f7nt2ruj3mbeu6o73t8e880/family1/7ac05b904c0749aeab8b326b9cd2ac02 into 6334f697e0e777acd41789a8c25bc8c9/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/6c35c9ae65184281be840e2a8cc2b5bc_SeqId_44_ - updating store file list. 2023-03-08 11:54:34,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/6c35c9ae65184281be840e2a8cc2b5bc_SeqId_44_ into 6334f697e0e777acd41789a8c25bc8c9/family1 2023-03-08 11:54:34,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__sc1nt5ot65l0it2tph619ctu91uerq4himlroove4f7nt2ruj3mbeu6o73t8e880/family1/7ac05b904c0749aeab8b326b9cd2ac02 into 6334f697e0e777acd41789a8c25bc8c9/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/6c35c9ae65184281be840e2a8cc2b5bc_SeqId_44_) 2023-03-08 11:54:34,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__sc1nt5ot65l0it2tph619ctu91uerq4himlroove4f7nt2ruj3mbeu6o73t8e880/family1/7ac05b904c0749aeab8b326b9cd2ac02 2023-03-08 11:54:34,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 11:54:34,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 11:54:34,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 6334f697e0e777acd41789a8c25bc8c9/family1 is initiating minor compaction 2023-03-08 11:54:34,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6a14df50; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:34,857 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6334f697e0e777acd41789a8c25bc8c9/family1 in testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:54:34,857 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/c5e661044dae4fdcab213fe0c0041d3d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/4d22b4fc0bb04284af937e39fcd7b5e0, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/1ffbfbf70a864d239647cb05baa30e43, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/054b8ec5f9a84775b220e1da377a6ab0, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/834cb34757674c8a9779156b70bc7c36, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/9d6a481455244d019ffdaf3299875913, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/ac945b7508ef4f629d91185e155e1835, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/8ccdcb5f03334c5a888ed1116d5c1f82, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/348e07f2de97412abcd66a597543ddcb, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/6c35c9ae65184281be840e2a8cc2b5bc_SeqId_44_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp, totalSize=50.0 K 2023-03-08 11:54:34,857 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting c5e661044dae4fdcab213fe0c0041d3d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 11:54:34,857 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 4d22b4fc0bb04284af937e39fcd7b5e0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 11:54:34,857 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 1ffbfbf70a864d239647cb05baa30e43, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 11:54:34,857 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 054b8ec5f9a84775b220e1da377a6ab0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 11:54:34,857 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 834cb34757674c8a9779156b70bc7c36, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 11:54:34,857 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 9d6a481455244d019ffdaf3299875913, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 11:54:34,857 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting ac945b7508ef4f629d91185e155e1835, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 11:54:34,857 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 8ccdcb5f03334c5a888ed1116d5c1f82, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 11:54:34,858 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 348e07f2de97412abcd66a597543ddcb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 11:54:34,858 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 6c35c9ae65184281be840e2a8cc2b5bc_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 11:54:34,859 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:34,859 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:34,861 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512281e239a3d32ac473b8fad569da0102013 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512281e239a3d32ac473b8fad569da0102013 2023-03-08 11:54:34,862 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231f750eb13b10a4729a6aa83a28a350d08 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231f750eb13b10a4729a6aa83a28a350d08 2023-03-08 11:54:34,864 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101f4e9a6cc58a2470d83aebfc613913149 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101f4e9a6cc58a2470d83aebfc613913149 2023-03-08 11:54:34,865 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010312acdb423bda47d9bf3f034f52fc3e67 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010312acdb423bda47d9bf3f034f52fc3e67 2023-03-08 11:54:34,865 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-08 11:54:34,865 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 11:54:34,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:34,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:34,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:34,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:34,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:34,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:34,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:34,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:34,902 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6334f697e0e777acd41789a8c25bc8c9#family1#compaction#48 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:54:34,935 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120562b4427664f64671b1a48b7cfad9351f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120562b4427664f64671b1a48b7cfad9351f 2023-03-08 11:54:34,936 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:34,963 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1dfa06ccf82f429aa26662d1f36ffc8e first=Optional[r00] last=Optional[r2] 2023-03-08 11:54:34,964 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1dfa06ccf82f429aa26662d1f36ffc8e}] 2023-03-08 11:54:34,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1dfa06ccf82f429aa26662d1f36ffc8e for inclusion in 6334f697e0e777acd41789a8c25bc8c9/family1 2023-03-08 11:54:34,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 11:54:34,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:54:34,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:34,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1dfa06ccf82f429aa26662d1f36ffc8e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jokdveiev89ktfr7dgtsga7bdk51pcjrv674i44ib7ei0ds0usntr3e2mnsr4is/family1/1dfa06ccf82f429aa26662d1f36ffc8e 2023-03-08 11:54:34,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jokdveiev89ktfr7dgtsga7bdk51pcjrv674i44ib7ei0ds0usntr3e2mnsr4is/family1/1dfa06ccf82f429aa26662d1f36ffc8e as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/c3816406e1df4876addf5b22a51a4392_SeqId_46_ 2023-03-08 11:54:34,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jokdveiev89ktfr7dgtsga7bdk51pcjrv674i44ib7ei0ds0usntr3e2mnsr4is/family1/1dfa06ccf82f429aa26662d1f36ffc8e into 6334f697e0e777acd41789a8c25bc8c9/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/c3816406e1df4876addf5b22a51a4392_SeqId_46_ - updating store file list. 2023-03-08 11:54:34,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/c3816406e1df4876addf5b22a51a4392_SeqId_46_ into 6334f697e0e777acd41789a8c25bc8c9/family1 2023-03-08 11:54:34,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jokdveiev89ktfr7dgtsga7bdk51pcjrv674i44ib7ei0ds0usntr3e2mnsr4is/family1/1dfa06ccf82f429aa26662d1f36ffc8e into 6334f697e0e777acd41789a8c25bc8c9/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/c3816406e1df4876addf5b22a51a4392_SeqId_46_) 2023-03-08 11:54:34,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jokdveiev89ktfr7dgtsga7bdk51pcjrv674i44ib7ei0ds0usntr3e2mnsr4is/family1/1dfa06ccf82f429aa26662d1f36ffc8e 2023-03-08 11:54:34,979 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:34,979 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:34,981 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113008b4f56bc7bc47618c955afb68d2bcf3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113008b4f56bc7bc47618c955afb68d2bcf3 2023-03-08 11:54:34,982 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120118bd201aaa0d4f3abd3f2c97c2cbbb5a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120118bd201aaa0d4f3abd3f2c97c2cbbb5a 2023-03-08 11:54:34,983 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120529aa82910f4c4c96a9b77e1e963c2d9d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120529aa82910f4c4c96a9b77e1e963c2d9d 2023-03-08 11:54:34,983 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-08 11:54:34,983 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 11:54:34,983 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 11:54:34,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 11:54:34,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:34,984 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:54:34,986 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0320 sec 2023-03-08 11:54:35,087 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=256 (was 244) - Thread LEAK? -, OpenFileDescriptor=628 (was 587) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=189 (was 204), ProcessCount=170 (was 170), AvailableMemoryMB=7572 (was 7488) - AvailableMemoryMB LEAK? - 2023-03-08 11:54:35,099 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=256, OpenFileDescriptor=628, MaxFileDescriptor=60000, SystemLoadAverage=189, ProcessCount=170, AvailableMemoryMB=7571 2023-03-08 11:54:35,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 11:54:35,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:35,103 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:54:35,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-08 11:54:35,104 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:54:35,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 11:54:35,106 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:35,107 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d empty. 2023-03-08 11:54:35,108 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:35,108 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-08 11:54:35,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 11:54:35,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 11:54:35,517 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-08 11:54:35,518 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 836c4c9f42bf977b9123d7bbe0300d0d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:54:35,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 11:54:35,734 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/.tmp/family1/ab185e1300fd4ffc86ce528d18bbac0d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/ab185e1300fd4ffc86ce528d18bbac0d 2023-03-08 11:54:35,739 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 6334f697e0e777acd41789a8c25bc8c9/family1 of 6334f697e0e777acd41789a8c25bc8c9 into ab185e1300fd4ffc86ce528d18bbac0d(size=6.6 K), total size for store is 337.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:54:35,739 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:54:35,740 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678276474857; duration=0sec 2023-03-08 11:54:35,740 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:35,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/c5e661044dae4fdcab213fe0c0041d3d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/4d22b4fc0bb04284af937e39fcd7b5e0, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/1ffbfbf70a864d239647cb05baa30e43, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/054b8ec5f9a84775b220e1da377a6ab0, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/834cb34757674c8a9779156b70bc7c36, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/9d6a481455244d019ffdaf3299875913, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/ac945b7508ef4f629d91185e155e1835, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/8ccdcb5f03334c5a888ed1116d5c1f82, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/348e07f2de97412abcd66a597543ddcb, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/6c35c9ae65184281be840e2a8cc2b5bc_SeqId_44_] to archive 2023-03-08 11:54:35,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:35,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/c5e661044dae4fdcab213fe0c0041d3d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/c5e661044dae4fdcab213fe0c0041d3d 2023-03-08 11:54:35,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/4d22b4fc0bb04284af937e39fcd7b5e0 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/4d22b4fc0bb04284af937e39fcd7b5e0 2023-03-08 11:54:35,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/1ffbfbf70a864d239647cb05baa30e43 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/1ffbfbf70a864d239647cb05baa30e43 2023-03-08 11:54:35,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/054b8ec5f9a84775b220e1da377a6ab0 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/054b8ec5f9a84775b220e1da377a6ab0 2023-03-08 11:54:35,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/834cb34757674c8a9779156b70bc7c36 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/834cb34757674c8a9779156b70bc7c36 2023-03-08 11:54:35,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/9d6a481455244d019ffdaf3299875913 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/9d6a481455244d019ffdaf3299875913 2023-03-08 11:54:35,782 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/ac945b7508ef4f629d91185e155e1835 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/ac945b7508ef4f629d91185e155e1835 2023-03-08 11:54:35,783 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/8ccdcb5f03334c5a888ed1116d5c1f82 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/8ccdcb5f03334c5a888ed1116d5c1f82 2023-03-08 11:54:35,784 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/348e07f2de97412abcd66a597543ddcb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/348e07f2de97412abcd66a597543ddcb 2023-03-08 11:54:35,785 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/6c35c9ae65184281be840e2a8cc2b5bc_SeqId_44_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/family1/6c35c9ae65184281be840e2a8cc2b5bc_SeqId_44_ 2023-03-08 11:54:35,924 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:35,925 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 836c4c9f42bf977b9123d7bbe0300d0d, disabling compactions & flushes 2023-03-08 11:54:35,925 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:35,925 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:35,925 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. after waiting 0 ms 2023-03-08 11:54:35,925 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:35,925 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:35,925 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:35,927 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:54:35,928 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276475928"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276475928"}]},"ts":"1678276475928"} 2023-03-08 11:54:35,929 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 11:54:35,930 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:54:35,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276475930"}]},"ts":"1678276475930"} 2023-03-08 11:54:35,931 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-08 11:54:35,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, ASSIGN}] 2023-03-08 11:54:35,936 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, ASSIGN 2023-03-08 11:54:35,936 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:54:36,087 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=836c4c9f42bf977b9123d7bbe0300d0d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:36,088 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276476087"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276476087"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276476087"}]},"ts":"1678276476087"} 2023-03-08 11:54:36,090 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 836c4c9f42bf977b9123d7bbe0300d0d, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:36,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 11:54:36,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:36,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 836c4c9f42bf977b9123d7bbe0300d0d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:54:36,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:36,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:36,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:36,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:36,247 INFO [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:36,249 DEBUG [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:36,249 DEBUG [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:36,249 INFO [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 836c4c9f42bf977b9123d7bbe0300d0d columnFamilyName family1 2023-03-08 11:54:36,250 INFO [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] regionserver.HStore(310): Store=836c4c9f42bf977b9123d7bbe0300d0d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:36,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:36,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:36,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:36,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:54:36,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 836c4c9f42bf977b9123d7bbe0300d0d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69409800, jitterRate=0.03428661823272705}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:54:36,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:36,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d., pid=41, masterSystemTime=1678276476241 2023-03-08 11:54:36,257 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 11:54:36,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:36,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:36,258 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=836c4c9f42bf977b9123d7bbe0300d0d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:36,258 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276476258"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276476258"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276476258"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276476258"}]},"ts":"1678276476258"} 2023-03-08 11:54:36,261 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-08 11:54:36,261 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 836c4c9f42bf977b9123d7bbe0300d0d, server=jenkins-hbase4.apache.org,45551,1678276414946 in 170 msec 2023-03-08 11:54:36,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-08 11:54:36,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, ASSIGN in 327 msec 2023-03-08 11:54:36,263 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:54:36,263 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276476263"}]},"ts":"1678276476263"} 2023-03-08 11:54:36,264 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-08 11:54:36,268 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:54:36,270 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1670 sec 2023-03-08 11:54:36,535 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:37,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 11:54:37,211 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-08 11:54:37,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:37,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,236 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:37,236 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:37,236 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 11:54:37,236 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 11:54:37,236 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,237 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,238 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:37,238 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:37,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:37,238 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,238 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:37,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,239 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:37,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,239 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 11:54:37,239 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:37,239 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 11:54:37,240 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:37,240 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:37,240 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:37,240 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. started... 2023-03-08 11:54:37,240 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 836c4c9f42bf977b9123d7bbe0300d0d 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 11:54:37,265 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511306faaeba5f4fe480aba9507c237bbd471 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511306faaeba5f4fe480aba9507c237bbd471 2023-03-08 11:54:37,269 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/da5d7c51b0514ad8809013ffb5b285cc 2023-03-08 11:54:37,275 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/da5d7c51b0514ad8809013ffb5b285cc as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/da5d7c51b0514ad8809013ffb5b285cc 2023-03-08 11:54:37,280 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/da5d7c51b0514ad8809013ffb5b285cc, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-08 11:54:37,281 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 836c4c9f42bf977b9123d7bbe0300d0d in 41ms, sequenceid=5, compaction requested=false 2023-03-08 11:54:37,281 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:37,282 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:37,282 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:37,282 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:37,282 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,282 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 11:54:37,282 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 11:54:37,283 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:37,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:37,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:37,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:37,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:37,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 11:54:37,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2993b57a[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:37,285 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 11:54:37,286 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,288 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,288 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:37,289 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,289 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(162): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,289 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,289 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:37,289 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 11:54:37,289 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 11:54:37,289 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:37,291 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,291 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 11:54:37,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,291 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:37,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:37,291 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 11:54:37,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:37,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:37,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:37,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:37,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:37,293 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 11:54:37,293 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 11:54:37,293 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:37,293 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:37,293 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,293 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:37,295 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,295 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:37,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:37,295 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:37,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:37,295 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:37,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:37,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:37,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,297 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:37,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:37,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:37,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:37,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:37,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:37,299 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:37,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:37,299 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:37,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:37,299 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:37,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:37,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,300 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,300 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,300 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,400 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:37,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:37,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:37,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:37,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,407 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:37,407 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:37,407 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 11:54:37,407 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 11:54:37,407 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,407 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,409 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,409 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:37,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:37,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:37,410 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:37,410 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:37,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:37,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,410 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,411 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 11:54:37,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:37,411 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 11:54:37,411 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:37,411 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 11:54:37,411 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:37,411 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:37,411 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:37,411 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. started... 2023-03-08 11:54:37,411 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 836c4c9f42bf977b9123d7bbe0300d0d 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 11:54:37,825 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511282112cdcebe4246c18998510b16b1ef32 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511282112cdcebe4246c18998510b16b1ef32 2023-03-08 11:54:38,231 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/f6562e52ba384b8b9911a6ee3a60247b 2023-03-08 11:54:38,235 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/f6562e52ba384b8b9911a6ee3a60247b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/f6562e52ba384b8b9911a6ee3a60247b 2023-03-08 11:54:38,240 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/f6562e52ba384b8b9911a6ee3a60247b, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 11:54:38,241 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 836c4c9f42bf977b9123d7bbe0300d0d in 830ms, sequenceid=9, compaction requested=false 2023-03-08 11:54:38,241 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:38,241 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:38,241 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:38,241 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:38,241 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,241 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 11:54:38,241 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 11:54:38,243 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,243 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:38,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:38,243 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,243 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:38,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:38,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:38,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:38,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 11:54:38,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22cc6728[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:38,245 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 11:54:38,245 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,246 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,246 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 11:54:38,246 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 11:54:38,246 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:38,246 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,246 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:38,247 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 11:54:38,247 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,247 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:38,247 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 11:54:38,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:38,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:38,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:38,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:38,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:38,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:38,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 11:54:38,250 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 11:54:38,250 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:38,250 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:38,250 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,250 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:38,252 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,252 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:38,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:38,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,252 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:38,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:38,253 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:38,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:38,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:38,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,254 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:38,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,258 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,259 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,259 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,259 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:38,259 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:38,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:38,259 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:38,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:38,259 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:38,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:38,259 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,259 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,259 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,259 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:38,259 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:38,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:38,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:38,359 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:38,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:38,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:38,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:38,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,366 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:38,366 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:38,366 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 11:54:38,366 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 11:54:38,367 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,367 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,368 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,368 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:38,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:38,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:38,368 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:38,368 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:38,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:38,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 11:54:38,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:38,370 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 11:54:38,370 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:38,370 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 11:54:38,370 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:38,370 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:38,370 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:38,370 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. started... 2023-03-08 11:54:38,370 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 836c4c9f42bf977b9123d7bbe0300d0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:38,784 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120157b88234d89846c3be14af3a125b5fb9 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120157b88234d89846c3be14af3a125b5fb9 2023-03-08 11:54:39,189 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/7a186ce6804a449b84f0708dd260c5ad 2023-03-08 11:54:39,195 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/7a186ce6804a449b84f0708dd260c5ad as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7a186ce6804a449b84f0708dd260c5ad 2023-03-08 11:54:39,199 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7a186ce6804a449b84f0708dd260c5ad, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 11:54:39,200 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 836c4c9f42bf977b9123d7bbe0300d0d in 830ms, sequenceid=13, compaction requested=true 2023-03-08 11:54:39,200 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:39,200 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 11:54:39,200 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 11:54:39,203 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,203 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:39,203 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,203 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:39,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:39,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:39,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 11:54:39,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c18f1d5[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:39,205 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 11:54:39,205 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,206 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 11:54:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 11:54:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:39,206 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,206 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:39,208 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,208 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 11:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,208 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:39,208 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 11:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:39,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 11:54:39,210 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 11:54:39,210 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:39,210 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:39,210 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,210 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:39,212 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,212 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:39,212 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:39,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:39,212 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:39,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,213 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:39,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,216 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,216 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,216 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:39,216 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:39,216 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:39,216 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:39,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 11:54:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:39,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:39,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:39,216 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,217 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,217 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,217 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:39,217 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:39,217 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:39,317 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:39,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:39,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:39,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:39,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,324 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:39,324 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:39,324 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 11:54:39,324 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 11:54:39,324 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,324 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,326 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,326 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:39,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:39,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:39,326 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:39,326 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:39,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:39,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 11:54:39,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:39,327 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 11:54:39,327 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:39,327 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 11:54:39,328 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:39,328 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:39,328 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:39,328 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. started... 2023-03-08 11:54:39,328 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 836c4c9f42bf977b9123d7bbe0300d0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:39,740 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205fbe2f22de8be4463a0667990f03f1849 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205fbe2f22de8be4463a0667990f03f1849 2023-03-08 11:54:40,145 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/54b90736b21247b28b60f29b71cfa91b 2023-03-08 11:54:40,150 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/54b90736b21247b28b60f29b71cfa91b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/54b90736b21247b28b60f29b71cfa91b 2023-03-08 11:54:40,154 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/54b90736b21247b28b60f29b71cfa91b, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 11:54:40,154 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 836c4c9f42bf977b9123d7bbe0300d0d in 826ms, sequenceid=17, compaction requested=true 2023-03-08 11:54:40,155 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:40,155 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:40,155 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:40,155 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:40,155 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,155 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 11:54:40,155 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 11:54:40,156 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,156 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:40,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:40,156 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,157 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:40,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:40,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:40,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:40,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 11:54:40,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4fec0b81[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:40,158 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 11:54:40,158 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,159 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,160 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 11:54:40,160 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 11:54:40,160 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:40,160 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,160 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:40,161 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 11:54:40,161 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,161 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:40,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,161 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 11:54:40,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:40,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:40,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:40,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:40,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:40,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:40,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 11:54:40,163 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 11:54:40,163 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:40,163 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:40,163 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,163 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:40,165 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,165 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:40,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:40,165 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:40,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:40,165 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:40,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:40,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:40,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,166 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:40,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,168 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:40,168 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:40,168 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:40,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:40,168 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,168 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,168 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:40,168 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,168 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,168 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,168 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,168 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:40,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:40,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:40,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:40,169 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:40,169 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:40,269 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:40,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:40,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:40,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:40,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,276 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:40,276 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:40,276 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 11:54:40,276 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 11:54:40,276 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,276 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,279 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,279 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:40,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:40,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:40,279 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:40,279 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:40,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,279 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:40,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,280 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 11:54:40,280 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:40,280 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 11:54:40,280 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:40,280 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 11:54:40,281 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:40,281 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:40,281 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:40,281 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. started... 2023-03-08 11:54:40,281 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 836c4c9f42bf977b9123d7bbe0300d0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:40,695 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c83eecf983684f48a7e721fd9c7d254b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c83eecf983684f48a7e721fd9c7d254b 2023-03-08 11:54:41,101 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/7215b17228904d8885841f3544be7fa3 2023-03-08 11:54:41,106 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/7215b17228904d8885841f3544be7fa3 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7215b17228904d8885841f3544be7fa3 2023-03-08 11:54:41,111 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7215b17228904d8885841f3544be7fa3, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 11:54:41,112 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 836c4c9f42bf977b9123d7bbe0300d0d in 830ms, sequenceid=21, compaction requested=true 2023-03-08 11:54:41,112 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:41,112 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:41,112 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:41,112 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:41,112 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,112 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 11:54:41,112 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 11:54:41,116 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,116 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:41,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:41,117 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,117 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:41,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:41,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:41,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:41,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 11:54:41,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7be04f63[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:41,118 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 11:54:41,118 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,119 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,119 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 11:54:41,119 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 11:54:41,119 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:41,119 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,120 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:41,121 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,121 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 11:54:41,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:41,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:41,121 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:41,122 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 11:54:41,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:41,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:41,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:41,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:41,123 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 11:54:41,123 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 11:54:41,124 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:41,124 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:41,124 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,124 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:41,125 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,125 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:41,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:41,125 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:41,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:41,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:41,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:41,125 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:41,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,127 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:41,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,129 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,129 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:41,129 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,129 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:41,129 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:41,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:41,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:41,129 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,129 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:41,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:41,129 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 11:54:41,130 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,130 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:41,130 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:41,130 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:41,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:41,230 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:41,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:41,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:41,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:41,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,237 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:41,237 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:41,237 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 11:54:41,237 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 11:54:41,237 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,237 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,239 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:41,239 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:41,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:41,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,239 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:41,239 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:41,239 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:41,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,240 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,240 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 11:54:41,240 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:41,240 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 11:54:41,240 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:41,240 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 11:54:41,240 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:41,240 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:41,240 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:41,240 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. started... 2023-03-08 11:54:41,240 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 836c4c9f42bf977b9123d7bbe0300d0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:41,656 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123184c38e1b930a4fbab08a9dfbd6f53434 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123184c38e1b930a4fbab08a9dfbd6f53434 2023-03-08 11:54:41,774 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:54:42,061 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/418bdb62f4bf40c09ebb5c1f715985a6 2023-03-08 11:54:42,066 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/418bdb62f4bf40c09ebb5c1f715985a6 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/418bdb62f4bf40c09ebb5c1f715985a6 2023-03-08 11:54:42,070 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/418bdb62f4bf40c09ebb5c1f715985a6, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 11:54:42,071 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 836c4c9f42bf977b9123d7bbe0300d0d in 831ms, sequenceid=25, compaction requested=true 2023-03-08 11:54:42,071 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:42,071 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:42,071 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:42,071 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:42,071 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,071 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 11:54:42,071 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 11:54:42,074 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,074 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:42,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:42,074 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,074 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:42,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:42,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:42,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:42,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 11:54:42,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a548210[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:42,076 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 11:54:42,076 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,077 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,077 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 11:54:42,077 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 11:54:42,077 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,077 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:42,077 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:42,078 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,078 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 11:54:42,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:42,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:42,079 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:42,079 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 11:54:42,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:42,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:42,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:42,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:42,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 11:54:42,081 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 11:54:42,081 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:42,081 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:42,081 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,081 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:42,083 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,083 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,083 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:42,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:42,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:42,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:42,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:42,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:42,084 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:42,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,085 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:42,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,088 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,088 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,088 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,088 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:42,088 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:42,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:42,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:42,088 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,088 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:42,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 11:54:42,088 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:42,089 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,089 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:42,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:42,089 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:42,089 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:42,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:42,189 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:42,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:42,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:42,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:42,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,196 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:42,196 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:42,196 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 11:54:42,196 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 11:54:42,196 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,196 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,198 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,198 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:42,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:42,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:42,198 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:42,198 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:42,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:42,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,199 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,199 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 11:54:42,199 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:42,199 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 11:54:42,199 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:42,199 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 11:54:42,199 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:42,199 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:42,199 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:42,199 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. started... 2023-03-08 11:54:42,200 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 836c4c9f42bf977b9123d7bbe0300d0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:42,612 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010190ddccefd458485c9f2d1be3c3164586 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010190ddccefd458485c9f2d1be3c3164586 2023-03-08 11:54:43,017 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/11f8a2aee8c24578a771cc553ddfb5d5 2023-03-08 11:54:43,022 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/11f8a2aee8c24578a771cc553ddfb5d5 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/11f8a2aee8c24578a771cc553ddfb5d5 2023-03-08 11:54:43,027 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/11f8a2aee8c24578a771cc553ddfb5d5, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 11:54:43,028 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 836c4c9f42bf977b9123d7bbe0300d0d in 829ms, sequenceid=29, compaction requested=true 2023-03-08 11:54:43,028 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:43,028 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:43,028 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:43,028 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:43,028 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,028 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 11:54:43,028 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 11:54:43,029 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,029 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:43,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:43,030 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,030 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:43,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:43,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:43,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:43,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 11:54:43,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1795ad6[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:43,031 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 11:54:43,031 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,033 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,033 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 11:54:43,033 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 11:54:43,033 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:43,033 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,033 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:43,036 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 11:54:43,036 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,036 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:43,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,036 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 11:54:43,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:43,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:43,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:43,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:43,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:43,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:43,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 11:54:43,038 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 11:54:43,038 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:43,038 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:43,038 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,038 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:43,042 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,042 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:43,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,042 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:43,042 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:43,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:43,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:43,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,043 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:43,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,045 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,045 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:43,045 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:43,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:43,045 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,045 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:43,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:43,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:43,045 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:43,046 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,046 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:43,046 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:43,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:43,046 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:43,046 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,046 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,146 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:43,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:43,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:43,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:43,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,153 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:43,153 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:43,153 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 11:54:43,153 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 11:54:43,154 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,154 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,156 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,156 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:43,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:43,157 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,157 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:43,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:43,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,157 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,157 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 11:54:43,158 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,158 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 11:54:43,158 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:43,158 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 11:54:43,158 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:43,158 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:43,158 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:43,158 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. started... 2023-03-08 11:54:43,158 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 836c4c9f42bf977b9123d7bbe0300d0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:43,570 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103620f722e8e4b4087b178cff2400934ae to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103620f722e8e4b4087b178cff2400934ae 2023-03-08 11:54:43,976 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/a313beeaa77942d690549ce427e1cb0b 2023-03-08 11:54:43,980 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/a313beeaa77942d690549ce427e1cb0b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/a313beeaa77942d690549ce427e1cb0b 2023-03-08 11:54:43,984 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/a313beeaa77942d690549ce427e1cb0b, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 11:54:43,985 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 836c4c9f42bf977b9123d7bbe0300d0d in 827ms, sequenceid=33, compaction requested=true 2023-03-08 11:54:43,985 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:43,985 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 11:54:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 11:54:43,987 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,987 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:43,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:43,987 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,987 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:43,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:43,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 11:54:43,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e66e97b[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:43,989 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 11:54:43,989 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,990 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,990 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 11:54:43,990 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,990 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 11:54:43,990 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:43,990 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:43,991 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 11:54:43,991 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:43,991 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,991 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 11:54:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:43,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:43,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 11:54:43,993 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 11:54:43,993 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:43,993 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:43,993 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,993 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:43,996 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,996 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,996 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:43,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:43,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:43,996 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:43,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:43,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:43,997 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:43,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:43,999 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:44,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:44,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:44,000 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:43,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:43,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:44,000 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,000 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:44,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 11:54:44,000 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:44,000 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,000 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:44,100 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:44,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:44,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:44,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:44,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,107 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:44,107 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:44,107 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 11:54:44,107 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 11:54:44,108 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,108 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,109 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:44,109 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:44,109 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,109 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:44,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 11:54:44,110 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,110 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 11:54:44,110 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:44,110 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 11:54:44,110 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:44,110 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:44,111 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:44,111 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. started... 2023-03-08 11:54:44,111 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 836c4c9f42bf977b9123d7bbe0300d0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:44,526 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090764ef85b5f6514bfaa98fcf57c372f9bf to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015090764ef85b5f6514bfaa98fcf57c372f9bf 2023-03-08 11:54:44,932 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/8fe12c49eb584b0e9ca4198270567a0b 2023-03-08 11:54:44,937 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/8fe12c49eb584b0e9ca4198270567a0b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/8fe12c49eb584b0e9ca4198270567a0b 2023-03-08 11:54:44,941 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/8fe12c49eb584b0e9ca4198270567a0b, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 11:54:44,942 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 836c4c9f42bf977b9123d7bbe0300d0d in 831ms, sequenceid=37, compaction requested=true 2023-03-08 11:54:44,942 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:44,942 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:44,942 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:44,942 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:44,942 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,942 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 11:54:44,943 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 11:54:44,944 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,944 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:44,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:44,944 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,944 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:44,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:44,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:44,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:44,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 11:54:44,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57043083[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:44,946 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 11:54:44,946 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,947 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,948 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 11:54:44,948 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 11:54:44,948 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,948 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:44,948 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:44,949 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,949 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 11:54:44,949 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:44,949 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 11:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:44,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:44,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:44,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:44,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 11:54:44,951 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 11:54:44,951 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:44,951 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:44,951 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,951 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:44,953 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,953 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:44,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,953 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:44,954 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:44,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:44,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:44,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,955 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,960 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,960 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:44,960 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,961 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:44,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 11:54:44,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:44,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:44,961 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:44,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:44,961 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:44,961 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:44,961 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,961 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:44,961 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:44,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:44,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:44,961 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:44,961 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,061 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:45,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:45,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:45,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:45,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,068 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:45,068 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:45,068 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 11:54:45,068 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 11:54:45,069 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,069 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,070 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,070 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:45,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:45,070 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,070 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,071 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,071 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,071 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 11:54:45,071 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 11:54:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:45,071 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 11:54:45,072 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:45,072 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:45,072 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:45,072 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. started... 2023-03-08 11:54:45,072 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 836c4c9f42bf977b9123d7bbe0300d0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:45,484 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511200fbc9c1cabbc4de48fd3c2e653ab6de2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511200fbc9c1cabbc4de48fd3c2e653ab6de2 2023-03-08 11:54:45,890 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/1ec478eae3e049afb5092db5022f49e5 2023-03-08 11:54:45,894 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/1ec478eae3e049afb5092db5022f49e5 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/1ec478eae3e049afb5092db5022f49e5 2023-03-08 11:54:45,898 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/1ec478eae3e049afb5092db5022f49e5, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 11:54:45,899 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 836c4c9f42bf977b9123d7bbe0300d0d in 827ms, sequenceid=41, compaction requested=true 2023-03-08 11:54:45,899 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:45,899 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:45,899 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:45,899 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:45,899 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,899 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 11:54:45,899 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 11:54:45,901 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,901 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:45,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:45,901 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,901 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:45,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:45,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:45,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:45,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 11:54:45,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@495e7c71[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:45,902 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 11:54:45,902 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,903 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,904 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 11:54:45,904 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,904 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:45,904 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 11:54:45,904 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:45,907 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 11:54:45,907 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,907 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:45,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:45,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:45,907 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 11:54:45,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:45,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:45,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:45,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:45,909 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 11:54:45,909 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 11:54:45,909 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:45,909 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:45,909 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,909 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:45,911 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,911 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:45,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:45,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,911 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:45,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:45,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:45,911 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:45,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:45,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,912 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:45,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,914 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,914 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:45,914 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:45,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 11:54:45,914 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:45,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:45,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:45,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:45,914 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:45,914 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,915 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:45,915 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,915 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,915 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:45,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:45,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:45,915 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:45,915 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:46,015 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:46,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:46,017 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:46,019 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 11:54:46,019 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 11:54:46,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:46,020 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277086020 2023-03-08 11:54:46,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:54:46,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 11:54:46,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:46,024 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x51d72abc to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:54:46,028 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c23cb0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:54:46,028 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:46,866 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010345078e81f47f40a9a154125b60482675 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010345078e81f47f40a9a154125b60482675 2023-03-08 11:54:46,867 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:46,879 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:54:46,880 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40712, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:54:46,886 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:54:46,888 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54808, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:54:46,901 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dcd88a38d94b4c01a05ddfb3fd63844d first=Optional[r3] last=Optional[r6] 2023-03-08 11:54:46,902 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dcd88a38d94b4c01a05ddfb3fd63844d}] 2023-03-08 11:54:46,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dcd88a38d94b4c01a05ddfb3fd63844d for inclusion in 836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:46,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 11:54:46,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:54:46,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:46,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dcd88a38d94b4c01a05ddfb3fd63844d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__il1kr888mv11rj0lpi3k2o8gqrjsmoevs8tbff7ss0loiu8aa8qug9vd52quvf78/family1/dcd88a38d94b4c01a05ddfb3fd63844d 2023-03-08 11:54:46,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__il1kr888mv11rj0lpi3k2o8gqrjsmoevs8tbff7ss0loiu8aa8qug9vd52quvf78/family1/dcd88a38d94b4c01a05ddfb3fd63844d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/923dc3ad7c0743008177b854bc50d665_SeqId_44_ 2023-03-08 11:54:46,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__il1kr888mv11rj0lpi3k2o8gqrjsmoevs8tbff7ss0loiu8aa8qug9vd52quvf78/family1/dcd88a38d94b4c01a05ddfb3fd63844d into 836c4c9f42bf977b9123d7bbe0300d0d/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/923dc3ad7c0743008177b854bc50d665_SeqId_44_ - updating store file list. 2023-03-08 11:54:46,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/923dc3ad7c0743008177b854bc50d665_SeqId_44_ into 836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:46,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__il1kr888mv11rj0lpi3k2o8gqrjsmoevs8tbff7ss0loiu8aa8qug9vd52quvf78/family1/dcd88a38d94b4c01a05ddfb3fd63844d into 836c4c9f42bf977b9123d7bbe0300d0d/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/923dc3ad7c0743008177b854bc50d665_SeqId_44_) 2023-03-08 11:54:46,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__il1kr888mv11rj0lpi3k2o8gqrjsmoevs8tbff7ss0loiu8aa8qug9vd52quvf78/family1/dcd88a38d94b4c01a05ddfb3fd63844d 2023-03-08 11:54:46,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 11:54:46,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 11:54:46,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 836c4c9f42bf977b9123d7bbe0300d0d/family1 is initiating minor compaction 2023-03-08 11:54:46,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@488ee0d7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:46,921 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 836c4c9f42bf977b9123d7bbe0300d0d/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:46,922 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/f6562e52ba384b8b9911a6ee3a60247b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7a186ce6804a449b84f0708dd260c5ad, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/54b90736b21247b28b60f29b71cfa91b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7215b17228904d8885841f3544be7fa3, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/418bdb62f4bf40c09ebb5c1f715985a6, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/11f8a2aee8c24578a771cc553ddfb5d5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/a313beeaa77942d690549ce427e1cb0b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/8fe12c49eb584b0e9ca4198270567a0b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/1ec478eae3e049afb5092db5022f49e5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/923dc3ad7c0743008177b854bc50d665_SeqId_44_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp, totalSize=50.4 K 2023-03-08 11:54:46,922 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting f6562e52ba384b8b9911a6ee3a60247b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 11:54:46,922 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 7a186ce6804a449b84f0708dd260c5ad, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 11:54:46,922 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 54b90736b21247b28b60f29b71cfa91b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 11:54:46,922 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 7215b17228904d8885841f3544be7fa3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 11:54:46,922 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 418bdb62f4bf40c09ebb5c1f715985a6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 11:54:46,922 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 11f8a2aee8c24578a771cc553ddfb5d5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 11:54:46,922 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting a313beeaa77942d690549ce427e1cb0b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 11:54:46,922 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 8fe12c49eb584b0e9ca4198270567a0b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 11:54:46,922 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 1ec478eae3e049afb5092db5022f49e5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 11:54:46,922 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 923dc3ad7c0743008177b854bc50d665_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 11:54:46,923 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:46,924 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:46,926 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c83eecf983684f48a7e721fd9c7d254b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c83eecf983684f48a7e721fd9c7d254b 2023-03-08 11:54:46,927 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123184c38e1b930a4fbab08a9dfbd6f53434 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123184c38e1b930a4fbab08a9dfbd6f53434 2023-03-08 11:54:46,928 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010190ddccefd458485c9f2d1be3c3164586 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010190ddccefd458485c9f2d1be3c3164586 2023-03-08 11:54:46,929 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103620f722e8e4b4087b178cff2400934ae to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103620f722e8e4b4087b178cff2400934ae 2023-03-08 11:54:46,929 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-08 11:54:46,929 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 11:54:46,930 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:46,930 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:46,930 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:46,930 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:46,930 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:46,930 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:46,930 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:46,930 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:46,952 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 836c4c9f42bf977b9123d7bbe0300d0d#family1#compaction#59 average throughput is 0.32 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:54:47,778 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/81b7ec9a2bb4421180075a6df275c221 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/81b7ec9a2bb4421180075a6df275c221 2023-03-08 11:54:47,783 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 836c4c9f42bf977b9123d7bbe0300d0d/family1 of 836c4c9f42bf977b9123d7bbe0300d0d into 81b7ec9a2bb4421180075a6df275c221(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:54:47,783 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:47,783 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678276486921; duration=0sec 2023-03-08 11:54:47,783 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:47,807 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205794db58f2e7049758ef9629e874c9fcf to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205794db58f2e7049758ef9629e874c9fcf 2023-03-08 11:54:47,808 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:47,832 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/63c9047321f54e2d858b814d6db5b2ff first=Optional[r00] last=Optional[r2] 2023-03-08 11:54:47,832 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/63c9047321f54e2d858b814d6db5b2ff}] 2023-03-08 11:54:47,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/63c9047321f54e2d858b814d6db5b2ff for inclusion in 836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:47,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 11:54:47,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:54:47,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:47,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/63c9047321f54e2d858b814d6db5b2ff to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__s5q5ner8rbhoqejhia9oelbppidaqdlr9flq7hldb56d42f2kcj2jg7l9ng0eadm/family1/63c9047321f54e2d858b814d6db5b2ff 2023-03-08 11:54:47,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__s5q5ner8rbhoqejhia9oelbppidaqdlr9flq7hldb56d42f2kcj2jg7l9ng0eadm/family1/63c9047321f54e2d858b814d6db5b2ff as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/56f36fd9b21649cb98293ac8d8b57638_SeqId_47_ 2023-03-08 11:54:47,840 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__s5q5ner8rbhoqejhia9oelbppidaqdlr9flq7hldb56d42f2kcj2jg7l9ng0eadm/family1/63c9047321f54e2d858b814d6db5b2ff into 836c4c9f42bf977b9123d7bbe0300d0d/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/56f36fd9b21649cb98293ac8d8b57638_SeqId_47_ - updating store file list. 2023-03-08 11:54:47,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/56f36fd9b21649cb98293ac8d8b57638_SeqId_47_ into 836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:47,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__s5q5ner8rbhoqejhia9oelbppidaqdlr9flq7hldb56d42f2kcj2jg7l9ng0eadm/family1/63c9047321f54e2d858b814d6db5b2ff into 836c4c9f42bf977b9123d7bbe0300d0d/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/56f36fd9b21649cb98293ac8d8b57638_SeqId_47_) 2023-03-08 11:54:47,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__s5q5ner8rbhoqejhia9oelbppidaqdlr9flq7hldb56d42f2kcj2jg7l9ng0eadm/family1/63c9047321f54e2d858b814d6db5b2ff 2023-03-08 11:54:47,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 11:54:47,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 11:54:47,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 836c4c9f42bf977b9123d7bbe0300d0d/family1 is initiating minor compaction (all files) 2023-03-08 11:54:47,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@562868c9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:47,845 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 836c4c9f42bf977b9123d7bbe0300d0d/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:47,845 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/da5d7c51b0514ad8809013ffb5b285cc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/81b7ec9a2bb4421180075a6df275c221, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/56f36fd9b21649cb98293ac8d8b57638_SeqId_47_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp, totalSize=390.1 K 2023-03-08 11:54:47,846 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting da5d7c51b0514ad8809013ffb5b285cc, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838541 2023-03-08 11:54:47,846 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 81b7ec9a2bb4421180075a6df275c221, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238541 2023-03-08 11:54:47,847 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 56f36fd9b21649cb98293ac8d8b57638_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878838541 2023-03-08 11:54:47,847 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:47,847 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:47,849 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511306faaeba5f4fe480aba9507c237bbd471 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511306faaeba5f4fe480aba9507c237bbd471 2023-03-08 11:54:47,850 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120157b88234d89846c3be14af3a125b5fb9 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120157b88234d89846c3be14af3a125b5fb9 2023-03-08 11:54:47,851 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205fbe2f22de8be4463a0667990f03f1849 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205fbe2f22de8be4463a0667990f03f1849 2023-03-08 11:54:47,852 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-08 11:54:47,852 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 11:54:47,852 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:47,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:47,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:47,853 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 11:54:47,855 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8350 sec 2023-03-08 11:54:47,874 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 836c4c9f42bf977b9123d7bbe0300d0d#family1#compaction#60 average throughput is 9.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:54:47,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/f6562e52ba384b8b9911a6ee3a60247b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7a186ce6804a449b84f0708dd260c5ad, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/54b90736b21247b28b60f29b71cfa91b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7215b17228904d8885841f3544be7fa3, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/418bdb62f4bf40c09ebb5c1f715985a6, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/11f8a2aee8c24578a771cc553ddfb5d5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/a313beeaa77942d690549ce427e1cb0b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/8fe12c49eb584b0e9ca4198270567a0b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/1ec478eae3e049afb5092db5022f49e5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/923dc3ad7c0743008177b854bc50d665_SeqId_44_] to archive 2023-03-08 11:54:47,886 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:47,888 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/f6562e52ba384b8b9911a6ee3a60247b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/f6562e52ba384b8b9911a6ee3a60247b 2023-03-08 11:54:47,891 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7a186ce6804a449b84f0708dd260c5ad to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7a186ce6804a449b84f0708dd260c5ad 2023-03-08 11:54:47,895 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/54b90736b21247b28b60f29b71cfa91b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/54b90736b21247b28b60f29b71cfa91b 2023-03-08 11:54:47,897 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7215b17228904d8885841f3544be7fa3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/7215b17228904d8885841f3544be7fa3 2023-03-08 11:54:47,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/418bdb62f4bf40c09ebb5c1f715985a6 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/418bdb62f4bf40c09ebb5c1f715985a6 2023-03-08 11:54:47,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/11f8a2aee8c24578a771cc553ddfb5d5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/11f8a2aee8c24578a771cc553ddfb5d5 2023-03-08 11:54:47,905 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/a313beeaa77942d690549ce427e1cb0b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/a313beeaa77942d690549ce427e1cb0b 2023-03-08 11:54:47,907 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/8fe12c49eb584b0e9ca4198270567a0b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/8fe12c49eb584b0e9ca4198270567a0b 2023-03-08 11:54:47,909 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/1ec478eae3e049afb5092db5022f49e5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/1ec478eae3e049afb5092db5022f49e5 2023-03-08 11:54:47,910 DEBUG [hfile_cleaner-dir-scan-pool-1] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-03-08 11:54:47,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/923dc3ad7c0743008177b854bc50d665_SeqId_44_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/923dc3ad7c0743008177b854bc50d665_SeqId_44_ 2023-03-08 11:54:47,919 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/.tmp/family1/2ebbea2b472e46a7b05d12571d08dcb2 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/2ebbea2b472e46a7b05d12571d08dcb2 2023-03-08 11:54:47,924 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 836c4c9f42bf977b9123d7bbe0300d0d/family1 of 836c4c9f42bf977b9123d7bbe0300d0d into 2ebbea2b472e46a7b05d12571d08dcb2(size=193.3 K), total size for store is 193.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:54:47,924 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:47,924 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1678276487845; duration=0sec 2023-03-08 11:54:47,925 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:54:47,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 11:54:47,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:47,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/da5d7c51b0514ad8809013ffb5b285cc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/81b7ec9a2bb4421180075a6df275c221, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/56f36fd9b21649cb98293ac8d8b57638_SeqId_47_] to archive 2023-03-08 11:54:47,970 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:47,972 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/da5d7c51b0514ad8809013ffb5b285cc to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/da5d7c51b0514ad8809013ffb5b285cc 2023-03-08 11:54:47,973 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/81b7ec9a2bb4421180075a6df275c221 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/81b7ec9a2bb4421180075a6df275c221 2023-03-08 11:54:47,974 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/56f36fd9b21649cb98293ac8d8b57638_SeqId_47_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/56f36fd9b21649cb98293ac8d8b57638_SeqId_47_ 2023-03-08 11:54:48,363 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 11:54:48,364 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-08 11:54:48,364 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 11:54:48,367 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-08 11:54:48,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, REOPEN/MOVE}] 2023-03-08 11:54:48,370 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, REOPEN/MOVE 2023-03-08 11:54:48,370 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=836c4c9f42bf977b9123d7bbe0300d0d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:48,370 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276488370"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276488370"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276488370"}]},"ts":"1678276488370"} 2023-03-08 11:54:48,372 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 836c4c9f42bf977b9123d7bbe0300d0d, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:48,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:48,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 836c4c9f42bf977b9123d7bbe0300d0d, disabling compactions & flushes 2023-03-08 11:54:48,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:48,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:48,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. after waiting 0 ms 2023-03-08 11:54:48,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:48,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 11:54:48,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:48,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:48,530 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 836c4c9f42bf977b9123d7bbe0300d0d to self. 2023-03-08 11:54:48,532 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:48,532 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=836c4c9f42bf977b9123d7bbe0300d0d, regionState=CLOSED 2023-03-08 11:54:48,532 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276488532"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276488532"}]},"ts":"1678276488532"} 2023-03-08 11:54:48,535 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-08 11:54:48,535 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 836c4c9f42bf977b9123d7bbe0300d0d, server=jenkins-hbase4.apache.org,45551,1678276414946 in 162 msec 2023-03-08 11:54:48,535 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=true 2023-03-08 11:54:48,686 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=836c4c9f42bf977b9123d7bbe0300d0d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:48,686 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276488686"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276488686"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276488686"}]},"ts":"1678276488686"} 2023-03-08 11:54:48,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 836c4c9f42bf977b9123d7bbe0300d0d, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:48,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:48,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 836c4c9f42bf977b9123d7bbe0300d0d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:54:48,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:48,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:48,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:48,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:48,844 INFO [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:48,845 DEBUG [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:48,845 DEBUG [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:48,846 INFO [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 836c4c9f42bf977b9123d7bbe0300d0d columnFamilyName family1 2023-03-08 11:54:48,850 DEBUG [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/2ebbea2b472e46a7b05d12571d08dcb2 2023-03-08 11:54:48,851 INFO [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] regionserver.HStore(310): Store=836c4c9f42bf977b9123d7bbe0300d0d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:48,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:48,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:48,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:48,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 836c4c9f42bf977b9123d7bbe0300d0d; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72849027, jitterRate=0.08553509414196014}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:54:48,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:48,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d., pid=47, masterSystemTime=1678276488839 2023-03-08 11:54:48,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:48,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:48,859 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=836c4c9f42bf977b9123d7bbe0300d0d, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:48,859 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276488859"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276488859"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276488859"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276488859"}]},"ts":"1678276488859"} 2023-03-08 11:54:48,862 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-08 11:54:48,862 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 836c4c9f42bf977b9123d7bbe0300d0d, server=jenkins-hbase4.apache.org,45551,1678276414946 in 172 msec 2023-03-08 11:54:48,864 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-08 11:54:48,864 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, REOPEN/MOVE in 493 msec 2023-03-08 11:54:48,866 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-08 11:54:48,866 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 497 msec 2023-03-08 11:54:48,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 914 msec 2023-03-08 11:54:48,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-08 11:54:48,870 INFO [Listener at localhost/36211] compactions.TestMobCompactor(278): alter status finished 2023-03-08 11:54:48,870 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:48,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 11:54:48,872 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 11:54:48,873 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:48,873 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277088873 2023-03-08 11:54:48,873 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:54:48,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 11:54:48,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:48,874 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x19b81e04 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:54:48,880 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@80e963f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:54:48,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:48,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:48,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:48,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:48,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:49,707 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280ad6ea2912404aa7a4016f031239d7f8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511280ad6ea2912404aa7a4016f031239d7f8 2023-03-08 11:54:49,708 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:49,721 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:54:49,722 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40722, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:54:49,728 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:54:49,730 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54818, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:54:49,738 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ea88fd257142422bb29b8b3b24f88457 first=Optional[r05] last=Optional[r8] 2023-03-08 11:54:49,739 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ea88fd257142422bb29b8b3b24f88457}] 2023-03-08 11:54:49,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ea88fd257142422bb29b8b3b24f88457 for inclusion in 836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:49,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 11:54:49,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:54:49,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:49,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ea88fd257142422bb29b8b3b24f88457 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__b14pp3lko1dik3vqpfjpp9dcoumbo5obmfrpepc0d6rrep444uqiofpmj1qbbr80/family1/ea88fd257142422bb29b8b3b24f88457 2023-03-08 11:54:49,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__b14pp3lko1dik3vqpfjpp9dcoumbo5obmfrpepc0d6rrep444uqiofpmj1qbbr80/family1/ea88fd257142422bb29b8b3b24f88457 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/aed38aa82e6544c0b03de86cead034b0_SeqId_53_ 2023-03-08 11:54:49,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__b14pp3lko1dik3vqpfjpp9dcoumbo5obmfrpepc0d6rrep444uqiofpmj1qbbr80/family1/ea88fd257142422bb29b8b3b24f88457 into 836c4c9f42bf977b9123d7bbe0300d0d/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/aed38aa82e6544c0b03de86cead034b0_SeqId_53_ - updating store file list. 2023-03-08 11:54:49,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/aed38aa82e6544c0b03de86cead034b0_SeqId_53_ into 836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:49,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__b14pp3lko1dik3vqpfjpp9dcoumbo5obmfrpepc0d6rrep444uqiofpmj1qbbr80/family1/ea88fd257142422bb29b8b3b24f88457 into 836c4c9f42bf977b9123d7bbe0300d0d/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/aed38aa82e6544c0b03de86cead034b0_SeqId_53_) 2023-03-08 11:54:49,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__b14pp3lko1dik3vqpfjpp9dcoumbo5obmfrpepc0d6rrep444uqiofpmj1qbbr80/family1/ea88fd257142422bb29b8b3b24f88457 2023-03-08 11:54:49,755 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:49,756 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:54:49,757 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511200fbc9c1cabbc4de48fd3c2e653ab6de2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511200fbc9c1cabbc4de48fd3c2e653ab6de2 2023-03-08 11:54:49,758 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511282112cdcebe4246c18998510b16b1ef32 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511282112cdcebe4246c18998510b16b1ef32 2023-03-08 11:54:49,758 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-08 11:54:49,758 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 11:54:49,759 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:49,759 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:49,759 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:49,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:49,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:49,760 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 11:54:49,762 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 889 msec 2023-03-08 11:54:49,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 11:54:49,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:50,229 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-08 11:54:50,230 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 11:54:50,230 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-08 11:54:50,232 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-08 11:54:50,235 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, REOPEN/MOVE}] 2023-03-08 11:54:50,235 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, REOPEN/MOVE 2023-03-08 11:54:50,236 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=836c4c9f42bf977b9123d7bbe0300d0d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:50,236 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276490236"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276490236"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276490236"}]},"ts":"1678276490236"} 2023-03-08 11:54:50,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 836c4c9f42bf977b9123d7bbe0300d0d, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:50,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:50,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 836c4c9f42bf977b9123d7bbe0300d0d, disabling compactions & flushes 2023-03-08 11:54:50,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:50,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:50,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. after waiting 0 ms 2023-03-08 11:54:50,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:50,395 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-08 11:54:50,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:50,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:50,396 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 836c4c9f42bf977b9123d7bbe0300d0d to self. 2023-03-08 11:54:50,398 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:50,398 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=836c4c9f42bf977b9123d7bbe0300d0d, regionState=CLOSED 2023-03-08 11:54:50,398 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276490398"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276490398"}]},"ts":"1678276490398"} 2023-03-08 11:54:50,401 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-08 11:54:50,401 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 836c4c9f42bf977b9123d7bbe0300d0d, server=jenkins-hbase4.apache.org,45551,1678276414946 in 161 msec 2023-03-08 11:54:50,401 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=true 2023-03-08 11:54:50,552 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=836c4c9f42bf977b9123d7bbe0300d0d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:50,552 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276490552"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276490552"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276490552"}]},"ts":"1678276490552"} 2023-03-08 11:54:50,554 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 836c4c9f42bf977b9123d7bbe0300d0d, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:50,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:50,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 836c4c9f42bf977b9123d7bbe0300d0d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:54:50,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:50,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:50,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:50,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:50,709 INFO [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:50,710 DEBUG [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:50,710 DEBUG [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1 2023-03-08 11:54:50,711 INFO [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 836c4c9f42bf977b9123d7bbe0300d0d columnFamilyName family1 2023-03-08 11:54:50,715 DEBUG [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/2ebbea2b472e46a7b05d12571d08dcb2 2023-03-08 11:54:50,718 DEBUG [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/family1/aed38aa82e6544c0b03de86cead034b0_SeqId_53_ 2023-03-08 11:54:50,718 INFO [StoreOpener-836c4c9f42bf977b9123d7bbe0300d0d-1] regionserver.HStore(310): Store=836c4c9f42bf977b9123d7bbe0300d0d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:50,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:50,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:50,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:54:50,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 836c4c9f42bf977b9123d7bbe0300d0d; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72702415, jitterRate=0.08335040509700775}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:54:50,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:54:50,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d., pid=53, masterSystemTime=1678276490705 2023-03-08 11:54:50,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:50,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:54:50,726 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=836c4c9f42bf977b9123d7bbe0300d0d, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:50,726 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678276490725"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276490725"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276490725"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276490725"}]},"ts":"1678276490725"} 2023-03-08 11:54:50,728 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-08 11:54:50,728 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 836c4c9f42bf977b9123d7bbe0300d0d, server=jenkins-hbase4.apache.org,45551,1678276414946 in 173 msec 2023-03-08 11:54:50,729 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-08 11:54:50,729 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=836c4c9f42bf977b9123d7bbe0300d0d, REOPEN/MOVE in 493 msec 2023-03-08 11:54:50,731 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-08 11:54:50,731 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 498 msec 2023-03-08 11:54:50,732 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 912 msec 2023-03-08 11:54:50,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-08 11:54:50,734 INFO [Listener at localhost/36211] compactions.TestMobCompactor(278): alter status finished 2023-03-08 11:54:50,735 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 11:54:50,736 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 11:54:50,736 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 11:54:50,737 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:50,737 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277090737 2023-03-08 11:54:50,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:54:50,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 11:54:50,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:50,738 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x7e1557cf to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:54:50,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d6995c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:54:50,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:50,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:50,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:50,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:50,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:50,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:50,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:50,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 11:54:50,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 11:54:50,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:54:50,745 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 11:54:50,746 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 10 msec 2023-03-08 11:54:50,791 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=248 (was 256), OpenFileDescriptor=626 (was 628), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=169 (was 189), ProcessCount=170 (was 170), AvailableMemoryMB=7557 (was 7571) 2023-03-08 11:54:50,802 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=248, OpenFileDescriptor=626, MaxFileDescriptor=60000, SystemLoadAverage=169, ProcessCount=170, AvailableMemoryMB=7556 2023-03-08 11:54:50,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 11:54:50,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:50,807 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:54:50,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-08 11:54:50,807 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:54:50,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 11:54:50,808 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:54:50,809 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989 empty. 2023-03-08 11:54:50,809 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:54:50,809 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-08 11:54:50,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 11:54:51,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 11:54:51,217 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-08 11:54:51,218 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2821201dfecad47e22ed1f74a6c77989, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:54:51,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 11:54:51,624 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:51,624 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing 2821201dfecad47e22ed1f74a6c77989, disabling compactions & flushes 2023-03-08 11:54:51,624 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:51,624 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:51,625 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. after waiting 0 ms 2023-03-08 11:54:51,625 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:51,625 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:51,625 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:54:51,627 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:54:51,628 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276491627"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276491627"}]},"ts":"1678276491627"} 2023-03-08 11:54:51,629 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 11:54:51,629 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:54:51,630 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276491630"}]},"ts":"1678276491630"} 2023-03-08 11:54:51,631 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-08 11:54:51,633 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, ASSIGN}] 2023-03-08 11:54:51,635 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, ASSIGN 2023-03-08 11:54:51,635 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:54:51,786 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=2821201dfecad47e22ed1f74a6c77989, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:51,787 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276491786"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276491786"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276491786"}]},"ts":"1678276491786"} 2023-03-08 11:54:51,788 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 2821201dfecad47e22ed1f74a6c77989, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:54:51,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 11:54:51,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:51,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2821201dfecad47e22ed1f74a6c77989, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:54:51,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:54:51,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:54:51,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:54:51,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:54:51,944 INFO [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:54:51,946 DEBUG [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:54:51,946 DEBUG [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:54:51,946 INFO [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2821201dfecad47e22ed1f74a6c77989 columnFamilyName family1 2023-03-08 11:54:51,947 INFO [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] regionserver.HStore(310): Store=2821201dfecad47e22ed1f74a6c77989/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:54:51,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:54:51,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:54:51,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:54:51,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:54:51,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2821201dfecad47e22ed1f74a6c77989; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62931350, jitterRate=-0.0622498095035553}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:54:51,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:54:51,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989., pid=57, masterSystemTime=1678276491940 2023-03-08 11:54:51,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:51,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:51,956 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=2821201dfecad47e22ed1f74a6c77989, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:51,956 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276491956"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276491956"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276491956"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276491956"}]},"ts":"1678276491956"} 2023-03-08 11:54:51,958 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-08 11:54:51,958 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 2821201dfecad47e22ed1f74a6c77989, server=jenkins-hbase4.apache.org,45551,1678276414946 in 169 msec 2023-03-08 11:54:51,960 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-08 11:54:51,960 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, ASSIGN in 325 msec 2023-03-08 11:54:51,960 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:54:51,960 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276491960"}]},"ts":"1678276491960"} 2023-03-08 11:54:51,961 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-08 11:54:51,968 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:54:51,969 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1630 sec 2023-03-08 11:54:52,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 11:54:52,913 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-08 11:54:52,954 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:52,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:52,956 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:52,956 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:52,956 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 11:54:52,957 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 11:54:52,957 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:52,957 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:52,958 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:52,958 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:52,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:52,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:52,959 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:52,959 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:52,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:52,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:52,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:52,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:52,960 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:52,960 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:52,960 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 11:54:52,960 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:52,960 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 11:54:52,960 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:52,960 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:52,960 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:52,960 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. started... 2023-03-08 11:54:52,961 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2821201dfecad47e22ed1f74a6c77989 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 11:54:53,391 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130bad05f75b3664895ae50a738307dcc60 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130bad05f75b3664895ae50a738307dcc60 2023-03-08 11:54:53,575 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:54:53,575 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x19b81e04 to 127.0.0.1:51863 2023-03-08 11:54:53,575 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:54:53,575 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e1557cf to 127.0.0.1:51863 2023-03-08 11:54:53,575 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:54:53,575 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:54:53,575 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x51d72abc to 127.0.0.1:51863 2023-03-08 11:54:53,575 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:54:53,576 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:54:53,576 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x65e0b25c to 127.0.0.1:51863 2023-03-08 11:54:53,576 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:54:53,576 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:54:53,576 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0034b74e to 127.0.0.1:51863 2023-03-08 11:54:53,576 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:54:53,795 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/7c56d6c0975f4c0e80faea11643dc62f 2023-03-08 11:54:53,801 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/7c56d6c0975f4c0e80faea11643dc62f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/7c56d6c0975f4c0e80faea11643dc62f 2023-03-08 11:54:53,805 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/7c56d6c0975f4c0e80faea11643dc62f, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-08 11:54:53,806 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2821201dfecad47e22ed1f74a6c77989 in 845ms, sequenceid=5, compaction requested=false 2023-03-08 11:54:53,807 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:53,807 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:54:53,808 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:53,808 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:53,808 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:53,808 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,808 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 11:54:53,808 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 11:54:53,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,810 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:53,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:53,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:53,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:53,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:53,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:53,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 11:54:53,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23a3066[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:53,812 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 11:54:53,812 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,813 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,813 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 11:54:53,813 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 11:54:53,813 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,813 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:53,813 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:53,815 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,815 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 11:54:53,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:53,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:53,815 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:53,815 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 11:54:53,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:53,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:53,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 11:54:53,817 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 11:54:53,817 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:53,817 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:53,817 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,817 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:53,820 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,820 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:53,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,820 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:53,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:53,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:53,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:53,821 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:53,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:53,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,822 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:53,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,826 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,826 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:53,826 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:53,826 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:53,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:53,826 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:53,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:53,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:53,827 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,827 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,827 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,827 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:53,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:53,827 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:53,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:53,827 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,827 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,927 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:53,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:53,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:53,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:53,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,934 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:53,934 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:53,934 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 11:54:53,934 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 11:54:53,934 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,934 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,935 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:53,935 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,936 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:53,936 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:53,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,937 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 11:54:53,937 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:53,937 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 11:54:53,937 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:53,937 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 11:54:53,937 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:53,937 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:53,937 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:53,937 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. started... 2023-03-08 11:54:53,937 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2821201dfecad47e22ed1f74a6c77989 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 11:54:54,088 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:54:54,350 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128db4b241738e94cb18d96dd08ffc2ce0d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128db4b241738e94cb18d96dd08ffc2ce0d 2023-03-08 11:54:54,538 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 11:54:54,755 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/fea0ba08fa874888a5aec2da8eecedd8 2023-03-08 11:54:54,759 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/fea0ba08fa874888a5aec2da8eecedd8 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fea0ba08fa874888a5aec2da8eecedd8 2023-03-08 11:54:54,763 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fea0ba08fa874888a5aec2da8eecedd8, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 11:54:54,764 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2821201dfecad47e22ed1f74a6c77989 in 827ms, sequenceid=9, compaction requested=false 2023-03-08 11:54:54,764 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:54:54,764 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:54,764 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:54,764 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:54,764 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,764 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 11:54:54,764 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 11:54:54,766 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,766 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:54,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:54,766 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,766 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:54,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:54,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:54,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:54,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 11:54:54,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b8e3c6b[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:54,768 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 11:54:54,768 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,769 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,769 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,769 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:54,769 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 11:54:54,769 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 11:54:54,769 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:54,770 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:54,770 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 11:54:54,770 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:54,770 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 11:54:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:54,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 11:54:54,772 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 11:54:54,772 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:54,772 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:54,772 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,772 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:54,774 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,774 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,774 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:54,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:54,774 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:54,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,775 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,778 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:54,778 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:54,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:54,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:54,778 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:54,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 11:54:54,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:54,778 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,778 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:54,779 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,779 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,779 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:54,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:54,779 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:54,779 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:54,779 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,779 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,879 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:54,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:54,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:54,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:54,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,886 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:54,886 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:54,886 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 11:54:54,886 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 11:54:54,886 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,886 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,887 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,887 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:54,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:54,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:54,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:54,888 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:54,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:54,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 11:54:54,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:54,889 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 11:54:54,889 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:54,889 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 11:54:54,889 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:54,889 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:54,889 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:54,889 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. started... 2023-03-08 11:54:54,889 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2821201dfecad47e22ed1f74a6c77989 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:54,900 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120103a7edadd9614dc3b0ae8d0547d7d836 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120103a7edadd9614dc3b0ae8d0547d7d836 2023-03-08 11:54:55,305 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/49e5fe8ccaeb4ca29cc4e9ceb5c7fb8d 2023-03-08 11:54:55,310 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/49e5fe8ccaeb4ca29cc4e9ceb5c7fb8d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/49e5fe8ccaeb4ca29cc4e9ceb5c7fb8d 2023-03-08 11:54:55,313 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/49e5fe8ccaeb4ca29cc4e9ceb5c7fb8d, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 11:54:55,314 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2821201dfecad47e22ed1f74a6c77989 in 425ms, sequenceid=13, compaction requested=true 2023-03-08 11:54:55,314 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:54:55,314 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:55,314 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:55,314 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:55,314 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,314 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 11:54:55,314 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 11:54:55,316 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,316 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:55,316 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,316 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 11:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d7897b1[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:55,317 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 11:54:55,317 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,320 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,320 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 11:54:55,320 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 11:54:55,320 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:55,320 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,320 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:55,321 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,321 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 11:54:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:55,321 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:55,321 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 11:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 11:54:55,323 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 11:54:55,323 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:55,324 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:55,324 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,324 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:55,325 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,325 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:55,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,325 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:55,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,326 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,327 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,329 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:55,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:55,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:55,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:55,329 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:55,429 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:55,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:55,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:55,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:55,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,435 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:55,435 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:55,435 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 11:54:55,436 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 11:54:55,436 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,436 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,438 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,438 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:55,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:55,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:55,438 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:55,438 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:55,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,439 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:55,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,439 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,439 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 11:54:55,439 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:55,439 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 11:54:55,439 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:55,439 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 11:54:55,440 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:55,440 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:55,440 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:55,440 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. started... 2023-03-08 11:54:55,440 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2821201dfecad47e22ed1f74a6c77989 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:55,852 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120596ce48b7c3934541be81883dcffd807d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120596ce48b7c3934541be81883dcffd807d 2023-03-08 11:54:56,257 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/b2e8af2bdd974e0ca83c976aa3bb5120 2023-03-08 11:54:56,262 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/b2e8af2bdd974e0ca83c976aa3bb5120 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/b2e8af2bdd974e0ca83c976aa3bb5120 2023-03-08 11:54:56,266 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/b2e8af2bdd974e0ca83c976aa3bb5120, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 11:54:56,267 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2821201dfecad47e22ed1f74a6c77989 in 827ms, sequenceid=17, compaction requested=true 2023-03-08 11:54:56,267 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:54:56,267 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:56,268 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:56,268 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:56,268 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,268 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 11:54:56,268 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 11:54:56,271 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,272 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:56,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:56,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 11:54:56,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@577fd39b[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:56,273 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 11:54:56,273 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,274 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,275 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 11:54:56,275 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 11:54:56,275 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:56,275 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,275 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:56,277 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,277 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 11:54:56,277 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:56,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:56,277 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 11:54:56,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:56,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:56,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:56,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:56,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:56,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 11:54:56,279 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 11:54:56,279 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:56,279 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:56,279 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,279 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:56,281 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,281 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,281 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:56,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:56,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:56,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,281 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:56,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:56,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,282 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:56,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,284 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:56,284 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:56,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:56,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:56,284 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,284 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:56,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:56,285 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:56,285 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,285 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,285 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:56,285 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,285 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,285 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,285 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:56,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:56,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:56,290 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 11:54:56,385 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:56,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:56,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:56,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:56,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,391 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:56,391 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:56,392 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 11:54:56,392 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 11:54:56,392 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,392 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,393 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,393 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:56,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:56,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:56,393 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:56,393 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,394 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,394 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 11:54:56,394 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 11:54:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 11:54:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:56,395 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:56,395 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. started... 2023-03-08 11:54:56,395 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2821201dfecad47e22ed1f74a6c77989 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:56,810 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228bf925111e2d043c09077811c67583315 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228bf925111e2d043c09077811c67583315 2023-03-08 11:54:57,215 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/0d85006ed26f4dd1b27b29b08a2c0faf 2023-03-08 11:54:57,220 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/0d85006ed26f4dd1b27b29b08a2c0faf as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/0d85006ed26f4dd1b27b29b08a2c0faf 2023-03-08 11:54:57,223 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/0d85006ed26f4dd1b27b29b08a2c0faf, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 11:54:57,224 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2821201dfecad47e22ed1f74a6c77989 in 829ms, sequenceid=21, compaction requested=true 2023-03-08 11:54:57,224 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:54:57,224 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:57,224 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:57,224 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:57,225 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,225 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 11:54:57,225 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 11:54:57,227 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,227 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:57,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:57,227 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,227 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:57,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:57,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:57,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:57,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 11:54:57,228 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 11:54:57,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4957012[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:57,229 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,230 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,230 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 11:54:57,230 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 11:54:57,230 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:57,230 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,230 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:57,231 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 11:54:57,231 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,231 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:57,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:57,231 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 11:54:57,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:57,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 11:54:57,234 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 11:54:57,234 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:57,234 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:57,234 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,234 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:57,235 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:57,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:57,235 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,235 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:57,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:57,236 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,237 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:57,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,244 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,244 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:57,244 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:57,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:57,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:57,244 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:57,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:57,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:57,244 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:57,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:57,244 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,244 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,244 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:57,245 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:57,244 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,245 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,245 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,345 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:57,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:57,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:57,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:57,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,351 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:57,351 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:57,351 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 11:54:57,351 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 11:54:57,352 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,352 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,353 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,353 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:57,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:57,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:57,353 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,353 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:57,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,354 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:57,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,354 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,354 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 11:54:57,354 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,354 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 11:54:57,354 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:57,354 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 11:54:57,354 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:57,354 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:57,354 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:57,354 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. started... 2023-03-08 11:54:57,355 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2821201dfecad47e22ed1f74a6c77989 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:57,367 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123140a2490b21eb42ddb210b399246de8e7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123140a2490b21eb42ddb210b399246de8e7 2023-03-08 11:54:57,772 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/c770f572d4f244b4b76e74c853b50602 2023-03-08 11:54:57,776 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/c770f572d4f244b4b76e74c853b50602 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c770f572d4f244b4b76e74c853b50602 2023-03-08 11:54:57,781 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c770f572d4f244b4b76e74c853b50602, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 11:54:57,782 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2821201dfecad47e22ed1f74a6c77989 in 427ms, sequenceid=25, compaction requested=true 2023-03-08 11:54:57,782 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:54:57,782 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 11:54:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 11:54:57,784 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,784 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:57,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:57,785 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,785 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:57,787 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 11:54:57,787 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e92698f[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:57,787 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 11:54:57,787 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,788 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,788 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 11:54:57,788 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 11:54:57,788 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:57,788 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,788 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:57,791 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,791 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 11:54:57,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,791 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:57,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:57,791 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 11:54:57,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:57,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:57,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:57,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:57,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:57,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 11:54:57,794 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 11:54:57,794 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:57,794 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:57,795 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,795 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:57,797 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,798 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:57,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:57,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,798 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:57,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:57,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:57,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:57,798 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:57,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,800 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:57,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,805 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,805 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,805 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,805 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:57,805 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:57,805 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:57,805 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:57,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:57,805 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:57,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:57,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 11:54:57,806 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:57,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:57,806 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,806 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:57,806 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:57,906 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:57,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:57,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:57,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:57,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,914 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:57,914 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:57,914 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 11:54:57,914 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 11:54:57,915 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,915 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,920 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,920 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:57,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:57,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:57,920 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:57,920 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 11:54:57,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:57,922 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 11:54:57,922 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:57,922 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 11:54:57,922 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:57,922 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:57,922 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:57,922 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. started... 2023-03-08 11:54:57,922 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2821201dfecad47e22ed1f74a6c77989 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:58,338 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601018f5f136f83d64829809297e259d41cc3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601018f5f136f83d64829809297e259d41cc3 2023-03-08 11:54:58,743 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/f55d8276ddb24290b3c1e223432d296b 2023-03-08 11:54:58,748 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/f55d8276ddb24290b3c1e223432d296b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/f55d8276ddb24290b3c1e223432d296b 2023-03-08 11:54:58,751 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/f55d8276ddb24290b3c1e223432d296b, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 11:54:58,752 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2821201dfecad47e22ed1f74a6c77989 in 830ms, sequenceid=29, compaction requested=true 2023-03-08 11:54:58,752 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:54:58,752 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:58,752 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:58,752 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:58,752 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,752 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 11:54:58,752 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 11:54:58,754 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,754 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:58,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:58,754 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,754 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:58,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:58,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:58,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:58,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 11:54:58,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47feaef3[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:58,756 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 11:54:58,756 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,757 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,757 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 11:54:58,757 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 11:54:58,757 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,757 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:58,757 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:58,759 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,759 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 11:54:58,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,759 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:58,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:58,759 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 11:54:58,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:58,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:58,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:58,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:58,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:58,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 11:54:58,762 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 11:54:58,762 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:58,762 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:58,762 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,762 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:58,763 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,763 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:58,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,763 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,764 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:58,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:58,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,765 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:58,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,767 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,767 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:58,767 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:58,767 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,767 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:58,767 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:58,767 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:58,767 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 11:54:58,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:58,767 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:58,768 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:58,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:58,768 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:58,768 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:58,768 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,768 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,868 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:58,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:58,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:58,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:58,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,875 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:58,875 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:58,875 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 11:54:58,875 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 11:54:58,875 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,875 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,876 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,876 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:58,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:58,876 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:58,877 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:58,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:58,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,877 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,878 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 11:54:58,878 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:58,878 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 11:54:58,878 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:58,878 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 11:54:58,878 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:58,878 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:58,878 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:58,878 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. started... 2023-03-08 11:54:58,878 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2821201dfecad47e22ed1f74a6c77989 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:54:59,293 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033a7cf41302b34766aef427d2690d41b7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601033a7cf41302b34766aef427d2690d41b7 2023-03-08 11:54:59,698 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/8b883001e26e49faac79af0fee11d960 2023-03-08 11:54:59,703 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/8b883001e26e49faac79af0fee11d960 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/8b883001e26e49faac79af0fee11d960 2023-03-08 11:54:59,707 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/8b883001e26e49faac79af0fee11d960, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 11:54:59,708 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2821201dfecad47e22ed1f74a6c77989 in 830ms, sequenceid=33, compaction requested=true 2023-03-08 11:54:59,708 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:54:59,708 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:59,708 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:54:59,709 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:54:59,709 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,709 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 11:54:59,709 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 11:54:59,716 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,716 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:59,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:59,716 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,716 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:54:59,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:59,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:59,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:59,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 11:54:59,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e344e9c[Count = 0] remaining members to acquire global barrier 2023-03-08 11:54:59,718 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 11:54:59,718 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,719 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,719 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 11:54:59,719 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 11:54:59,719 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,719 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:54:59,719 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:54:59,720 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:59,721 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 11:54:59,721 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:59,721 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 11:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:59,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:54:59,723 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 11:54:59,723 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 11:54:59,723 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:54:59,723 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:54:59,723 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,723 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:54:59,724 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,724 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:54:59,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:54:59,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,724 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:59,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:54:59,725 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:54:59,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,726 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:54:59,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:59,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,729 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:54:59,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,729 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:54:59,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:54:59,829 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:54:59,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:54:59,835 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:54:59,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 11:54:59,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,837 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:54:59,837 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:59,837 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 11:54:59,837 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 11:54:59,838 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,838 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,839 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,839 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:54:59,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:54:59,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:54:59,840 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:54:59,840 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:54:59,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,840 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:54:59,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,841 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,841 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 11:54:59,841 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:54:59,841 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 11:54:59,841 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:54:59,841 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 11:54:59,844 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:54:59,844 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:54:59,844 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:54:59,844 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. started... 2023-03-08 11:54:59,844 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2821201dfecad47e22ed1f74a6c77989 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:55:00,268 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078c3702b9e11f4254be601ad74ed06d59 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509078c3702b9e11f4254be601ad74ed06d59 2023-03-08 11:55:00,674 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/fe1b3cbead3a4073842ebe9ce1823ced 2023-03-08 11:55:00,679 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/fe1b3cbead3a4073842ebe9ce1823ced as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fe1b3cbead3a4073842ebe9ce1823ced 2023-03-08 11:55:00,684 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fe1b3cbead3a4073842ebe9ce1823ced, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 11:55:00,684 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2821201dfecad47e22ed1f74a6c77989 in 840ms, sequenceid=37, compaction requested=true 2023-03-08 11:55:00,685 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:00,685 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:00,685 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:55:00,685 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:55:00,685 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,685 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 11:55:00,685 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 11:55:00,686 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,686 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:00,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:00,687 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,687 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:00,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:00,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:00,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:00,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 11:55:00,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@410eb6a8[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:00,688 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 11:55:00,688 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,691 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,691 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 11:55:00,691 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,691 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:00,691 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 11:55:00,691 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:00,692 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 11:55:00,692 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,692 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:00,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:00,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:00,692 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 11:55:00,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:00,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:00,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:00,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:00,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 11:55:00,694 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 11:55:00,695 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:00,695 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:00,695 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,695 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:00,696 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,696 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:00,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:00,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,696 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:00,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,696 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:00,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:00,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:00,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:00,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,697 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:00,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,699 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,699 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,699 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:00,699 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:55:00,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:00,699 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,699 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,699 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,699 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:00,699 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:00,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:00,699 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:00,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 11:55:00,700 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:00,700 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:00,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:00,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:00,800 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:00,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:00,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:00,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 11:55:00,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,806 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:55:00,806 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:00,806 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 11:55:00,806 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 11:55:00,807 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,807 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,809 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,809 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:00,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:00,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:00,809 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:00,809 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:00,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,810 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:00,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,810 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,810 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 11:55:00,810 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:00,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 11:55:00,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:00,810 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 11:55:00,811 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:55:00,811 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:00,811 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:00,811 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. started... 2023-03-08 11:55:00,811 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2821201dfecad47e22ed1f74a6c77989 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:55:01,230 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112043a8a05e35734170ab6265a36fe9c107 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112043a8a05e35734170ab6265a36fe9c107 2023-03-08 11:55:01,637 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/ddd92d27c6ad44dcb59b7ed6aaa049f5 2023-03-08 11:55:01,642 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/ddd92d27c6ad44dcb59b7ed6aaa049f5 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/ddd92d27c6ad44dcb59b7ed6aaa049f5 2023-03-08 11:55:01,646 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/ddd92d27c6ad44dcb59b7ed6aaa049f5, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 11:55:01,647 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2821201dfecad47e22ed1f74a6c77989 in 836ms, sequenceid=41, compaction requested=true 2023-03-08 11:55:01,647 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:01,647 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:01,647 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:55:01,647 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:55:01,648 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,648 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 11:55:01,648 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 11:55:01,649 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,649 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:01,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:01,649 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,649 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:01,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:01,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:01,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:01,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 11:55:01,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cad7117[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:01,651 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 11:55:01,651 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,652 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 11:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 11:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:01,652 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,652 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:01,653 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,653 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 11:55:01,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:01,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:01,653 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:01,653 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 11:55:01,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:01,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:01,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:01,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:01,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 11:55:01,655 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 11:55:01,655 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:01,656 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:01,656 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,656 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:01,658 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,658 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:01,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:01,658 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:01,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:01,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:01,658 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:01,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,659 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:01,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,661 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,661 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,661 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:01,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:55:01,662 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:01,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:01,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:01,661 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:01,661 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:01,662 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:01,662 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,662 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:01,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 11:55:01,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:01,662 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:01,662 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:01,662 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,762 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:01,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:01,765 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:01,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 11:55:01,767 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 11:55:01,767 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277101767 2023-03-08 11:55:01,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:55:01,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:55:01,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 11:55:01,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:01,771 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x715c79f4 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:55:01,777 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4807323e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:55:01,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:02,614 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103da1284eac2db46d79977c5d891faecfe to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103da1284eac2db46d79977c5d891faecfe 2023-03-08 11:55:02,615 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:02,634 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:55:02,635 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41624, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:55:02,643 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:55:02,644 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42606, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:55:02,653 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2d26613416374fffa151c5b81c0c43b5 first=Optional[r3] last=Optional[r6] 2023-03-08 11:55:02,654 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2d26613416374fffa151c5b81c0c43b5}] 2023-03-08 11:55:02,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2d26613416374fffa151c5b81c0c43b5 for inclusion in 2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:55:02,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 11:55:02,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:55:02,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:02,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2d26613416374fffa151c5b81c0c43b5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7l6dvcdql4d0knfcah583kmedl7qljm2pmq4agi1s5frpuhv058sotgt4astbrqo/family1/2d26613416374fffa151c5b81c0c43b5 2023-03-08 11:55:02,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7l6dvcdql4d0knfcah583kmedl7qljm2pmq4agi1s5frpuhv058sotgt4astbrqo/family1/2d26613416374fffa151c5b81c0c43b5 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/00b1ae92b86f4c1e848347e0585212db_SeqId_44_ 2023-03-08 11:55:02,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7l6dvcdql4d0knfcah583kmedl7qljm2pmq4agi1s5frpuhv058sotgt4astbrqo/family1/2d26613416374fffa151c5b81c0c43b5 into 2821201dfecad47e22ed1f74a6c77989/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/00b1ae92b86f4c1e848347e0585212db_SeqId_44_ - updating store file list. 2023-03-08 11:55:02,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/00b1ae92b86f4c1e848347e0585212db_SeqId_44_ into 2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:55:02,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7l6dvcdql4d0knfcah583kmedl7qljm2pmq4agi1s5frpuhv058sotgt4astbrqo/family1/2d26613416374fffa151c5b81c0c43b5 into 2821201dfecad47e22ed1f74a6c77989/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/00b1ae92b86f4c1e848347e0585212db_SeqId_44_) 2023-03-08 11:55:02,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7l6dvcdql4d0knfcah583kmedl7qljm2pmq4agi1s5frpuhv058sotgt4astbrqo/family1/2d26613416374fffa151c5b81c0c43b5 2023-03-08 11:55:02,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 11:55:02,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 11:55:02,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 2821201dfecad47e22ed1f74a6c77989/family1 is initiating minor compaction 2023-03-08 11:55:02,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2126b5c9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:55:02,673 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2821201dfecad47e22ed1f74a6c77989/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:02,673 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fea0ba08fa874888a5aec2da8eecedd8, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/49e5fe8ccaeb4ca29cc4e9ceb5c7fb8d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/b2e8af2bdd974e0ca83c976aa3bb5120, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/0d85006ed26f4dd1b27b29b08a2c0faf, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c770f572d4f244b4b76e74c853b50602, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/f55d8276ddb24290b3c1e223432d296b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/8b883001e26e49faac79af0fee11d960, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fe1b3cbead3a4073842ebe9ce1823ced, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/ddd92d27c6ad44dcb59b7ed6aaa049f5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/00b1ae92b86f4c1e848347e0585212db_SeqId_44_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp, totalSize=50.3 K 2023-03-08 11:55:02,674 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting fea0ba08fa874888a5aec2da8eecedd8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 11:55:02,674 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 49e5fe8ccaeb4ca29cc4e9ceb5c7fb8d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 11:55:02,674 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting b2e8af2bdd974e0ca83c976aa3bb5120, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 11:55:02,674 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 0d85006ed26f4dd1b27b29b08a2c0faf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 11:55:02,674 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting c770f572d4f244b4b76e74c853b50602, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 11:55:02,674 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting f55d8276ddb24290b3c1e223432d296b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 11:55:02,674 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 8b883001e26e49faac79af0fee11d960, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 11:55:02,674 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting fe1b3cbead3a4073842ebe9ce1823ced, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 11:55:02,674 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting ddd92d27c6ad44dcb59b7ed6aaa049f5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 11:55:02,674 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 00b1ae92b86f4c1e848347e0585212db_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 11:55:02,675 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:02,676 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:02,678 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228bf925111e2d043c09077811c67583315 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228bf925111e2d043c09077811c67583315 2023-03-08 11:55:02,679 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123140a2490b21eb42ddb210b399246de8e7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123140a2490b21eb42ddb210b399246de8e7 2023-03-08 11:55:02,681 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601018f5f136f83d64829809297e259d41cc3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601018f5f136f83d64829809297e259d41cc3 2023-03-08 11:55:02,682 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601033a7cf41302b34766aef427d2690d41b7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601033a7cf41302b34766aef427d2690d41b7 2023-03-08 11:55:02,682 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-08 11:55:02,682 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 11:55:02,683 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:02,683 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:02,683 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:02,683 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:02,683 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:02,683 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:02,683 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:02,683 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:02,736 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2821201dfecad47e22ed1f74a6c77989#family1#compaction#71 average throughput is 0.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:02,767 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/a05b8a276f364f4f8748af8e317305ad as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/a05b8a276f364f4f8748af8e317305ad 2023-03-08 11:55:02,771 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2821201dfecad47e22ed1f74a6c77989/family1 of 2821201dfecad47e22ed1f74a6c77989 into a05b8a276f364f4f8748af8e317305ad(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:55:02,772 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:02,772 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678276502673; duration=0sec 2023-03-08 11:55:02,772 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:55:02,869 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fea0ba08fa874888a5aec2da8eecedd8, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/49e5fe8ccaeb4ca29cc4e9ceb5c7fb8d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/b2e8af2bdd974e0ca83c976aa3bb5120, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/0d85006ed26f4dd1b27b29b08a2c0faf, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c770f572d4f244b4b76e74c853b50602, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/f55d8276ddb24290b3c1e223432d296b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/8b883001e26e49faac79af0fee11d960, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fe1b3cbead3a4073842ebe9ce1823ced, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/ddd92d27c6ad44dcb59b7ed6aaa049f5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/00b1ae92b86f4c1e848347e0585212db_SeqId_44_] to archive 2023-03-08 11:55:02,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:02,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fea0ba08fa874888a5aec2da8eecedd8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fea0ba08fa874888a5aec2da8eecedd8 2023-03-08 11:55:02,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/49e5fe8ccaeb4ca29cc4e9ceb5c7fb8d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/49e5fe8ccaeb4ca29cc4e9ceb5c7fb8d 2023-03-08 11:55:02,874 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/b2e8af2bdd974e0ca83c976aa3bb5120 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/b2e8af2bdd974e0ca83c976aa3bb5120 2023-03-08 11:55:02,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/0d85006ed26f4dd1b27b29b08a2c0faf to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/0d85006ed26f4dd1b27b29b08a2c0faf 2023-03-08 11:55:02,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c770f572d4f244b4b76e74c853b50602 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c770f572d4f244b4b76e74c853b50602 2023-03-08 11:55:02,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/f55d8276ddb24290b3c1e223432d296b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/f55d8276ddb24290b3c1e223432d296b 2023-03-08 11:55:02,878 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/8b883001e26e49faac79af0fee11d960 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/8b883001e26e49faac79af0fee11d960 2023-03-08 11:55:02,879 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fe1b3cbead3a4073842ebe9ce1823ced to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/fe1b3cbead3a4073842ebe9ce1823ced 2023-03-08 11:55:02,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/ddd92d27c6ad44dcb59b7ed6aaa049f5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/ddd92d27c6ad44dcb59b7ed6aaa049f5 2023-03-08 11:55:02,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/00b1ae92b86f4c1e848347e0585212db_SeqId_44_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/00b1ae92b86f4c1e848347e0585212db_SeqId_44_ 2023-03-08 11:55:03,157 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f112e143c5fd427f9121ccbd7f0c6e58 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205f112e143c5fd427f9121ccbd7f0c6e58 2023-03-08 11:55:03,158 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:03,182 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a490a55bd10a4d0d899608481d6ba4f4 first=Optional[r00] last=Optional[r2] 2023-03-08 11:55:03,182 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a490a55bd10a4d0d899608481d6ba4f4}] 2023-03-08 11:55:03,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a490a55bd10a4d0d899608481d6ba4f4 for inclusion in 2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:55:03,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 11:55:03,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:55:03,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:03,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a490a55bd10a4d0d899608481d6ba4f4 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__qp1tsttoi34cmldcrncrpttk4jte79kfirkf6nem9jut5fjqtp52gut33bb1qctk/family1/a490a55bd10a4d0d899608481d6ba4f4 2023-03-08 11:55:03,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__qp1tsttoi34cmldcrncrpttk4jte79kfirkf6nem9jut5fjqtp52gut33bb1qctk/family1/a490a55bd10a4d0d899608481d6ba4f4 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c4db9b2de27c49319688ecf579c066b9_SeqId_47_ 2023-03-08 11:55:03,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__qp1tsttoi34cmldcrncrpttk4jte79kfirkf6nem9jut5fjqtp52gut33bb1qctk/family1/a490a55bd10a4d0d899608481d6ba4f4 into 2821201dfecad47e22ed1f74a6c77989/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c4db9b2de27c49319688ecf579c066b9_SeqId_47_ - updating store file list. 2023-03-08 11:55:03,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c4db9b2de27c49319688ecf579c066b9_SeqId_47_ into 2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:55:03,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__qp1tsttoi34cmldcrncrpttk4jte79kfirkf6nem9jut5fjqtp52gut33bb1qctk/family1/a490a55bd10a4d0d899608481d6ba4f4 into 2821201dfecad47e22ed1f74a6c77989/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c4db9b2de27c49319688ecf579c066b9_SeqId_47_) 2023-03-08 11:55:03,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__qp1tsttoi34cmldcrncrpttk4jte79kfirkf6nem9jut5fjqtp52gut33bb1qctk/family1/a490a55bd10a4d0d899608481d6ba4f4 2023-03-08 11:55:03,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 11:55:03,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 11:55:03,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 2821201dfecad47e22ed1f74a6c77989/family1 is initiating minor compaction (all files) 2023-03-08 11:55:03,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@60ccc43b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:55:03,199 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2821201dfecad47e22ed1f74a6c77989/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:03,199 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/7c56d6c0975f4c0e80faea11643dc62f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/a05b8a276f364f4f8748af8e317305ad, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c4db9b2de27c49319688ecf579c066b9_SeqId_47_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp, totalSize=388.1 K 2023-03-08 11:55:03,199 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 7c56d6c0975f4c0e80faea11643dc62f, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838541 2023-03-08 11:55:03,200 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting a05b8a276f364f4f8748af8e317305ad, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238541 2023-03-08 11:55:03,201 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting c4db9b2de27c49319688ecf579c066b9_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878838541 2023-03-08 11:55:03,201 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:03,201 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:03,203 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130bad05f75b3664895ae50a738307dcc60 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130bad05f75b3664895ae50a738307dcc60 2023-03-08 11:55:03,204 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120103a7edadd9614dc3b0ae8d0547d7d836 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120103a7edadd9614dc3b0ae8d0547d7d836 2023-03-08 11:55:03,205 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120596ce48b7c3934541be81883dcffd807d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120596ce48b7c3934541be81883dcffd807d 2023-03-08 11:55:03,205 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-08 11:55:03,205 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 11:55:03,206 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:03,206 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:03,206 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:55:03,207 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 11:55:03,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.4420 sec 2023-03-08 11:55:03,215 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2821201dfecad47e22ed1f74a6c77989#family1#compaction#72 average throughput is 25.66 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:03,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 11:55:03,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:03,294 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 11:55:03,295 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-08 11:55:03,295 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 11:55:03,297 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-08 11:55:03,300 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, REOPEN/MOVE}] 2023-03-08 11:55:03,301 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, REOPEN/MOVE 2023-03-08 11:55:03,301 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2821201dfecad47e22ed1f74a6c77989, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:03,302 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276503301"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276503301"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276503301"}]},"ts":"1678276503301"} 2023-03-08 11:55:03,303 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 2821201dfecad47e22ed1f74a6c77989, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:03,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:03,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2821201dfecad47e22ed1f74a6c77989, disabling compactions & flushes 2023-03-08 11:55:03,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:04,433 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/.tmp/family1/93fd8b79da6048ccb5cca666c6a35a34 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/93fd8b79da6048ccb5cca666c6a35a34 2023-03-08 11:55:04,437 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 2821201dfecad47e22ed1f74a6c77989/family1 of 2821201dfecad47e22ed1f74a6c77989 into 93fd8b79da6048ccb5cca666c6a35a34(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:04,437 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:04,437 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:04,438 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1678276503199; duration=1sec 2023-03-08 11:55:04,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:04,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. after waiting 0 ms 2023-03-08 11:55:04,438 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:55:04,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:04,442 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/7c56d6c0975f4c0e80faea11643dc62f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/a05b8a276f364f4f8748af8e317305ad, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c4db9b2de27c49319688ecf579c066b9_SeqId_47_] to archive 2023-03-08 11:55:04,442 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:04,444 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/7c56d6c0975f4c0e80faea11643dc62f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/7c56d6c0975f4c0e80faea11643dc62f 2023-03-08 11:55:04,445 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/a05b8a276f364f4f8748af8e317305ad to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/a05b8a276f364f4f8748af8e317305ad 2023-03-08 11:55:04,446 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c4db9b2de27c49319688ecf579c066b9_SeqId_47_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/c4db9b2de27c49319688ecf579c066b9_SeqId_47_ 2023-03-08 11:55:04,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 11:55:04,452 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:04,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:04,452 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 2821201dfecad47e22ed1f74a6c77989 to self. 2023-03-08 11:55:04,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:04,454 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2821201dfecad47e22ed1f74a6c77989, regionState=CLOSED 2023-03-08 11:55:04,454 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276504454"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276504454"}]},"ts":"1678276504454"} 2023-03-08 11:55:04,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-08 11:55:04,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 2821201dfecad47e22ed1f74a6c77989, server=jenkins-hbase4.apache.org,45551,1678276414946 in 1.1520 sec 2023-03-08 11:55:04,456 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=true 2023-03-08 11:55:04,607 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2821201dfecad47e22ed1f74a6c77989, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:04,607 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276504607"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276504607"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276504607"}]},"ts":"1678276504607"} 2023-03-08 11:55:04,608 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 2821201dfecad47e22ed1f74a6c77989, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:04,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:04,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2821201dfecad47e22ed1f74a6c77989, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:55:04,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:04,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:04,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:04,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:04,765 INFO [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:04,765 DEBUG [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:55:04,765 DEBUG [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:55:04,766 INFO [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2821201dfecad47e22ed1f74a6c77989 columnFamilyName family1 2023-03-08 11:55:04,770 DEBUG [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/93fd8b79da6048ccb5cca666c6a35a34 2023-03-08 11:55:04,770 INFO [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] regionserver.HStore(310): Store=2821201dfecad47e22ed1f74a6c77989/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:04,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:04,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:04,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:04,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2821201dfecad47e22ed1f74a6c77989; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64412872, jitterRate=-0.04017341136932373}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:55:04,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:04,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989., pid=63, masterSystemTime=1678276504760 2023-03-08 11:55:04,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:04,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:04,777 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2821201dfecad47e22ed1f74a6c77989, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:04,777 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276504777"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276504777"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276504777"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276504777"}]},"ts":"1678276504777"} 2023-03-08 11:55:04,779 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-08 11:55:04,779 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 2821201dfecad47e22ed1f74a6c77989, server=jenkins-hbase4.apache.org,45551,1678276414946 in 170 msec 2023-03-08 11:55:04,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-08 11:55:04,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, REOPEN/MOVE in 1.4790 sec 2023-03-08 11:55:04,783 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-08 11:55:04,783 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.4840 sec 2023-03-08 11:55:04,784 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5040 sec 2023-03-08 11:55:04,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-08 11:55:04,786 INFO [Listener at localhost/36211] compactions.TestMobCompactor(278): alter status finished 2023-03-08 11:55:04,787 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:04,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 11:55:04,788 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 11:55:04,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:55:04,789 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277104789 2023-03-08 11:55:04,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:55:04,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 11:55:04,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:04,790 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x3f49ad69 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:55:04,795 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1fec54de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:55:04,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:04,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:04,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:04,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:04,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:05,620 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c77435d0a7b846a9a46d9c0437195dbb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128c77435d0a7b846a9a46d9c0437195dbb 2023-03-08 11:55:05,621 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:05,639 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:55:05,641 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44990, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:55:05,647 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:55:05,648 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35496, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:55:05,657 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2e83885bebc745ee87bc311ee372f0d1 first=Optional[r05] last=Optional[r8] 2023-03-08 11:55:05,657 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2e83885bebc745ee87bc311ee372f0d1}] 2023-03-08 11:55:05,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2e83885bebc745ee87bc311ee372f0d1 for inclusion in 2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:55:05,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 11:55:05,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:55:05,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:05,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2e83885bebc745ee87bc311ee372f0d1 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__i6ibrlmlc10cj5en6b5cscahg7lvspmi0pt3ito3s2ahov5gctghmshq2dcv12kf/family1/2e83885bebc745ee87bc311ee372f0d1 2023-03-08 11:55:05,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__i6ibrlmlc10cj5en6b5cscahg7lvspmi0pt3ito3s2ahov5gctghmshq2dcv12kf/family1/2e83885bebc745ee87bc311ee372f0d1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/32e26042b5cc4d7faae485c7b2325610_SeqId_53_ 2023-03-08 11:55:05,667 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__i6ibrlmlc10cj5en6b5cscahg7lvspmi0pt3ito3s2ahov5gctghmshq2dcv12kf/family1/2e83885bebc745ee87bc311ee372f0d1 into 2821201dfecad47e22ed1f74a6c77989/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/32e26042b5cc4d7faae485c7b2325610_SeqId_53_ - updating store file list. 2023-03-08 11:55:05,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/32e26042b5cc4d7faae485c7b2325610_SeqId_53_ into 2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:55:05,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__i6ibrlmlc10cj5en6b5cscahg7lvspmi0pt3ito3s2ahov5gctghmshq2dcv12kf/family1/2e83885bebc745ee87bc311ee372f0d1 into 2821201dfecad47e22ed1f74a6c77989/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/32e26042b5cc4d7faae485c7b2325610_SeqId_53_) 2023-03-08 11:55:05,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__i6ibrlmlc10cj5en6b5cscahg7lvspmi0pt3ito3s2ahov5gctghmshq2dcv12kf/family1/2e83885bebc745ee87bc311ee372f0d1 2023-03-08 11:55:05,672 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:05,673 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:05,674 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112043a8a05e35734170ab6265a36fe9c107 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112043a8a05e35734170ab6265a36fe9c107 2023-03-08 11:55:05,675 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128db4b241738e94cb18d96dd08ffc2ce0d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128db4b241738e94cb18d96dd08ffc2ce0d 2023-03-08 11:55:05,675 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-08 11:55:05,675 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 11:55:05,675 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:05,675 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:05,675 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:05,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:05,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:55:05,676 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 11:55:05,678 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 890 msec 2023-03-08 11:55:05,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-08 11:55:05,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:06,133 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-08 11:55:06,134 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 11:55:06,134 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-08 11:55:06,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-08 11:55:06,138 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, REOPEN/MOVE}] 2023-03-08 11:55:06,138 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, REOPEN/MOVE 2023-03-08 11:55:06,139 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2821201dfecad47e22ed1f74a6c77989, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:06,139 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276506139"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276506139"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276506139"}]},"ts":"1678276506139"} 2023-03-08 11:55:06,140 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 2821201dfecad47e22ed1f74a6c77989, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:06,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:06,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2821201dfecad47e22ed1f74a6c77989, disabling compactions & flushes 2023-03-08 11:55:06,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:06,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:06,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. after waiting 0 ms 2023-03-08 11:55:06,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:06,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-08 11:55:06,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:06,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:06,298 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 2821201dfecad47e22ed1f74a6c77989 to self. 2023-03-08 11:55:06,299 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:06,299 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2821201dfecad47e22ed1f74a6c77989, regionState=CLOSED 2023-03-08 11:55:06,300 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276506299"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276506299"}]},"ts":"1678276506299"} 2023-03-08 11:55:06,302 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-08 11:55:06,302 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 2821201dfecad47e22ed1f74a6c77989, server=jenkins-hbase4.apache.org,45551,1678276414946 in 161 msec 2023-03-08 11:55:06,302 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=true 2023-03-08 11:55:06,452 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2821201dfecad47e22ed1f74a6c77989, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:06,453 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276506452"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276506452"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276506452"}]},"ts":"1678276506452"} 2023-03-08 11:55:06,454 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 2821201dfecad47e22ed1f74a6c77989, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:06,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:06,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2821201dfecad47e22ed1f74a6c77989, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:55:06,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:06,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:06,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:06,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:06,610 INFO [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:06,611 DEBUG [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:55:06,611 DEBUG [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1 2023-03-08 11:55:06,611 INFO [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2821201dfecad47e22ed1f74a6c77989 columnFamilyName family1 2023-03-08 11:55:06,616 DEBUG [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/32e26042b5cc4d7faae485c7b2325610_SeqId_53_ 2023-03-08 11:55:06,619 DEBUG [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/family1/93fd8b79da6048ccb5cca666c6a35a34 2023-03-08 11:55:06,619 INFO [StoreOpener-2821201dfecad47e22ed1f74a6c77989-1] regionserver.HStore(310): Store=2821201dfecad47e22ed1f74a6c77989/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:06,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:06,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:06,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:55:06,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2821201dfecad47e22ed1f74a6c77989; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74888462, jitterRate=0.11592504382133484}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:55:06,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:55:06,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989., pid=69, masterSystemTime=1678276506606 2023-03-08 11:55:06,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:06,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:55:06,626 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2821201dfecad47e22ed1f74a6c77989, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:06,626 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678276506626"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276506626"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276506626"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276506626"}]},"ts":"1678276506626"} 2023-03-08 11:55:06,628 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-08 11:55:06,628 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 2821201dfecad47e22ed1f74a6c77989, server=jenkins-hbase4.apache.org,45551,1678276414946 in 173 msec 2023-03-08 11:55:06,629 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-08 11:55:06,629 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2821201dfecad47e22ed1f74a6c77989, REOPEN/MOVE in 490 msec 2023-03-08 11:55:06,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-08 11:55:06,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 494 msec 2023-03-08 11:55:06,632 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 908 msec 2023-03-08 11:55:06,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-08 11:55:06,634 INFO [Listener at localhost/36211] compactions.TestMobCompactor(278): alter status finished 2023-03-08 11:55:06,636 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 11:55:06,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 11:55:06,637 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 11:55:06,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:55:06,638 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277106638 2023-03-08 11:55:06,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:55:06,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 11:55:06,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:06,640 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x7e2c90ca to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:55:06,644 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37b78ebf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:55:06,645 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:06,645 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:06,645 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:06,645 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:06,645 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:06,645 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:06,645 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:06,645 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 11:55:06,645 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 11:55:06,645 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:55:06,646 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 11:55:06,647 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 10 msec 2023-03-08 11:55:06,692 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=241 (was 248), OpenFileDescriptor=622 (was 626), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=164 (was 169), ProcessCount=170 (was 170), AvailableMemoryMB=7493 (was 7556) 2023-03-08 11:55:06,703 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=241, OpenFileDescriptor=622, MaxFileDescriptor=60000, SystemLoadAverage=164, ProcessCount=170, AvailableMemoryMB=7493 2023-03-08 11:55:06,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 11:55:06,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-08 11:55:06,708 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:55:06,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-08 11:55:06,709 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:55:06,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 11:55:06,710 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:06,710 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708 empty. 2023-03-08 11:55:06,711 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:06,711 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-08 11:55:06,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 11:55:07,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 11:55:07,118 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-08 11:55:07,119 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 964cc659f2b4048d9e6d9a05dea2a708, NAME => 'testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:55:07,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 11:55:07,525 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:07,525 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 964cc659f2b4048d9e6d9a05dea2a708, disabling compactions & flushes 2023-03-08 11:55:07,525 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:07,525 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:07,525 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. after waiting 0 ms 2023-03-08 11:55:07,525 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:07,525 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:07,525 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 964cc659f2b4048d9e6d9a05dea2a708: 2023-03-08 11:55:07,527 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:55:07,528 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678276507528"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276507528"}]},"ts":"1678276507528"} 2023-03-08 11:55:07,529 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 11:55:07,530 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:55:07,530 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276507530"}]},"ts":"1678276507530"} 2023-03-08 11:55:07,531 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-08 11:55:07,534 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=964cc659f2b4048d9e6d9a05dea2a708, ASSIGN}] 2023-03-08 11:55:07,535 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=964cc659f2b4048d9e6d9a05dea2a708, ASSIGN 2023-03-08 11:55:07,535 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=964cc659f2b4048d9e6d9a05dea2a708, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:55:07,686 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=964cc659f2b4048d9e6d9a05dea2a708, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:07,687 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678276507686"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276507686"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276507686"}]},"ts":"1678276507686"} 2023-03-08 11:55:07,688 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 964cc659f2b4048d9e6d9a05dea2a708, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:07,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 11:55:07,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:07,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 964cc659f2b4048d9e6d9a05dea2a708, NAME => 'testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:55:07,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-08 11:55:07,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-08 11:55:07,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:07,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:07,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:07,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:07,845 INFO [StoreOpener-964cc659f2b4048d9e6d9a05dea2a708-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:07,846 DEBUG [StoreOpener-964cc659f2b4048d9e6d9a05dea2a708-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1 2023-03-08 11:55:07,846 DEBUG [StoreOpener-964cc659f2b4048d9e6d9a05dea2a708-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1 2023-03-08 11:55:07,847 INFO [StoreOpener-964cc659f2b4048d9e6d9a05dea2a708-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 964cc659f2b4048d9e6d9a05dea2a708 columnFamilyName f1 2023-03-08 11:55:07,848 INFO [StoreOpener-964cc659f2b4048d9e6d9a05dea2a708-1] regionserver.HStore(310): Store=964cc659f2b4048d9e6d9a05dea2a708/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:07,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:07,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:07,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:07,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:55:07,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 964cc659f2b4048d9e6d9a05dea2a708; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74251091, jitterRate=0.10642747581005096}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:55:07,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 964cc659f2b4048d9e6d9a05dea2a708: 2023-03-08 11:55:07,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708., pid=73, masterSystemTime=1678276507839 2023-03-08 11:55:07,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:07,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:07,856 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=964cc659f2b4048d9e6d9a05dea2a708, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:07,856 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678276507855"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276507855"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276507855"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276507855"}]},"ts":"1678276507855"} 2023-03-08 11:55:07,858 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-08 11:55:07,858 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 964cc659f2b4048d9e6d9a05dea2a708, server=jenkins-hbase4.apache.org,45551,1678276414946 in 169 msec 2023-03-08 11:55:07,859 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-08 11:55:07,859 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=964cc659f2b4048d9e6d9a05dea2a708, ASSIGN in 325 msec 2023-03-08 11:55:07,860 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:55:07,860 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276507860"}]},"ts":"1678276507860"} 2023-03-08 11:55:07,861 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-08 11:55:07,864 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:55:07,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1580 sec 2023-03-08 11:55:08,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 11:55:08,815 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-08 11:55:08,815 DEBUG [Listener at localhost/36211] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-08 11:55:08,817 INFO [Listener at localhost/36211] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-08 11:55:08,820 INFO [Listener at localhost/36211] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-08 11:55:08,820 INFO [Listener at localhost/36211] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-08 11:55:08,820 INFO [Listener at localhost/36211] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-08 11:55:08,824 INFO [Listener at localhost/36211] regionserver.HRegion(2745): Flushing 964cc659f2b4048d9e6d9a05dea2a708 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-08 11:55:09,236 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308ff2beb264655491ba26033e85f683ed7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308ff2beb264655491ba26033e85f683ed7 2023-03-08 11:55:09,642 INFO [Listener at localhost/36211] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/.tmp/f1/1e6b27aadfe74a1c8879eafa80eb5ec4 2023-03-08 11:55:09,646 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/.tmp/f1/1e6b27aadfe74a1c8879eafa80eb5ec4 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4 2023-03-08 11:55:09,650 INFO [Listener at localhost/36211] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4, entries=1, sequenceid=5, filesize=4.9 K 2023-03-08 11:55:09,651 INFO [Listener at localhost/36211] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 964cc659f2b4048d9e6d9a05dea2a708 in 827ms, sequenceid=5, compaction requested=false 2023-03-08 11:55:09,651 DEBUG [Listener at localhost/36211] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-08 11:55:09,652 DEBUG [Listener at localhost/36211] regionserver.HRegion(2446): Flush status journal for 964cc659f2b4048d9e6d9a05dea2a708: 2023-03-08 11:55:09,653 INFO [Listener at localhost/36211] regionserver.HRegion(2745): Flushing 964cc659f2b4048d9e6d9a05dea2a708 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-08 11:55:09,980 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:55:10,064 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030821c036c9c9334a839fb8b2cc13e48017 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030821c036c9c9334a839fb8b2cc13e48017 2023-03-08 11:55:10,470 INFO [Listener at localhost/36211] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/.tmp/f1/21aeebf21de241179d71bd4e1aea2ddf 2023-03-08 11:55:10,475 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/.tmp/f1/21aeebf21de241179d71bd4e1aea2ddf as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf 2023-03-08 11:55:10,478 INFO [Listener at localhost/36211] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf, entries=1, sequenceid=9, filesize=4.9 K 2023-03-08 11:55:10,479 INFO [Listener at localhost/36211] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 964cc659f2b4048d9e6d9a05dea2a708 in 826ms, sequenceid=9, compaction requested=false 2023-03-08 11:55:10,479 DEBUG [Listener at localhost/36211] regionserver.HRegion(2446): Flush status journal for 964cc659f2b4048d9e6d9a05dea2a708: 2023-03-08 11:55:10,481 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-08 11:55:10,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 11:55:10,483 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 11:55:10,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:55:10,483 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277110483 2023-03-08 11:55:10,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 11:55:10,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-08 11:55:10,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-08 11:55:10,485 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x6fbf5bad to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:55:10,489 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6476e293, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:55:10,490 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testGetAfterCompaction' and column='f1' 2023-03-08 11:55:10,532 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,533 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,583 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,584 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,634 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,634 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,685 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,685 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,736 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,736 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,787 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,787 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,838 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,838 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,889 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,889 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,940 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,940 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,991 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:10,991 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,042 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,042 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,093 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,093 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,144 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,144 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,195 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,195 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,246 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,246 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,297 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,297 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,314 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308c465b8df418347cd91732945565251e5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308c465b8df418347cd91732945565251e5 2023-03-08 11:55:11,314 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testGetAfterCompaction' column='f1' 2023-03-08 11:55:11,327 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:55:11,328 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44998, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:55:11,334 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:55:11,336 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35508, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:55:11,345 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/ccd3c55933d9408c812edd435acd0b1f first=Optional[r1] last=Optional[r2] 2023-03-08 11:55:11,345 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{f1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/ccd3c55933d9408c812edd435acd0b1f}] 2023-03-08 11:55:11,348 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,348 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/ccd3c55933d9408c812edd435acd0b1f for inclusion in 964cc659f2b4048d9e6d9a05dea2a708/f1 2023-03-08 11:55:11,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-08 11:55:11,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:55:11,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 964cc659f2b4048d9e6d9a05dea2a708: 2023-03-08 11:55:11,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/ccd3c55933d9408c812edd435acd0b1f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testGetAfterCompaction__goo9slu2nnq82ldoqhgd2n1l9s5faba0vn9aifisg9e5n8nqkufg0sj9u88dhev5/f1/ccd3c55933d9408c812edd435acd0b1f 2023-03-08 11:55:11,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testGetAfterCompaction__goo9slu2nnq82ldoqhgd2n1l9s5faba0vn9aifisg9e5n8nqkufg0sj9u88dhev5/f1/ccd3c55933d9408c812edd435acd0b1f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_ 2023-03-08 11:55:11,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testGetAfterCompaction__goo9slu2nnq82ldoqhgd2n1l9s5faba0vn9aifisg9e5n8nqkufg0sj9u88dhev5/f1/ccd3c55933d9408c812edd435acd0b1f into 964cc659f2b4048d9e6d9a05dea2a708/f1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_ - updating store file list. 2023-03-08 11:55:11,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_ into 964cc659f2b4048d9e6d9a05dea2a708/f1 2023-03-08 11:55:11,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testGetAfterCompaction__goo9slu2nnq82ldoqhgd2n1l9s5faba0vn9aifisg9e5n8nqkufg0sj9u88dhev5/f1/ccd3c55933d9408c812edd435acd0b1f into 964cc659f2b4048d9e6d9a05dea2a708/f1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_) 2023-03-08 11:55:11,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testGetAfterCompaction__goo9slu2nnq82ldoqhgd2n1l9s5faba0vn9aifisg9e5n8nqkufg0sj9u88dhev5/f1/ccd3c55933d9408c812edd435acd0b1f 2023-03-08 11:55:11,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 964cc659f2b4048d9e6d9a05dea2a708/f1 is initiating minor compaction 2023-03-08 11:55:11,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6da69609; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:55:11,358 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 964cc659f2b4048d9e6d9a05dea2a708/f1 in testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:11,358 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/.tmp, totalSize=10.0 K 2023-03-08 11:55:11,358 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 21aeebf21de241179d71bd4e1aea2ddf, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 11:55:11,358 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 7472bc9ed21f467189538049718f3ad4_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 11:55:11,360 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testGetAfterCompaction' column='f1' 2023-03-08 11:55:11,361 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:11,362 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030821c036c9c9334a839fb8b2cc13e48017 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023030821c036c9c9334a839fb8b2cc13e48017 2023-03-08 11:55:11,364 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308ff2beb264655491ba26033e85f683ed7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308ff2beb264655491ba26033e85f683ed7 2023-03-08 11:55:11,364 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-08 11:55:11,364 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 11:55:11,364 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testGetAfterCompaction' and column='f1' 2023-03-08 11:55:11,364 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-08 11:55:11,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:55:11,366 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 11:55:11,366 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 964cc659f2b4048d9e6d9a05dea2a708#f1#compaction#75 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:11,367 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 885 msec 2023-03-08 11:55:11,400 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,400 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,400 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678276511309; access_time=1678276510500; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 11:55:11,403 DEBUG [Listener at localhost/36211] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 11:55:11,406 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45002, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 11:55:11,407 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45551] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:11,407 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-08 11:55:11,407 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 11:55:11,407 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45551] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 11:55:11,407 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45551] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. because compaction request was cancelled 2023-03-08 11:55:11,459 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,459 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,459 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678276511309; access_time=1678276510500; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 11:55:11,510 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,510 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,510 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678276511309; access_time=1678276510500; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 11:55:11,561 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,561 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,561 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678276511309; access_time=1678276510500; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 11:55:11,612 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,612 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,612 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678276511309; access_time=1678276510500; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 11:55:11,663 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,663 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,663 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678276511309; access_time=1678276510500; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 11:55:11,714 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,714 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,714 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678276511309; access_time=1678276510500; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 11:55:11,765 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,765 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,765 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678276511309; access_time=1678276510500; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 11:55:11,779 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/.tmp/f1/9c4b864881214d3086467dbd644c0119 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/9c4b864881214d3086467dbd644c0119 2023-03-08 11:55:11,784 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 964cc659f2b4048d9e6d9a05dea2a708/f1 of 964cc659f2b4048d9e6d9a05dea2a708 into 9c4b864881214d3086467dbd644c0119(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:55:11,784 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 964cc659f2b4048d9e6d9a05dea2a708: 2023-03-08 11:55:11,784 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1678276511358; duration=0sec 2023-03-08 11:55:11,784 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:55:11,816 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,816 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276510470; access_time=1678276509655; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,816 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678276511309; access_time=1678276510500; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 11:55:11,816 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/9c4b864881214d3086467dbd644c0119; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678276511774; access_time=1678276511363; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,863 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_] to archive 2023-03-08 11:55:11,864 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:11,866 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/21aeebf21de241179d71bd4e1aea2ddf 2023-03-08 11:55:11,867 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,867 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678276511309; access_time=1678276510500; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 11:55:11,867 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/9c4b864881214d3086467dbd644c0119; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678276511774; access_time=1678276511363; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,867 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/7472bc9ed21f467189538049718f3ad4_SeqId_12_ 2023-03-08 11:55:11,918 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678276509641; access_time=1678276508827; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:11,918 INFO [Listener at localhost/36211] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/9c4b864881214d3086467dbd644c0119; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678276511774; access_time=1678276511363; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 11:55:12,034 INFO [Listener at localhost/36211] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-08 11:55:12,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-03-08 11:55:12,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-08 11:55:12,047 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276512047"}]},"ts":"1678276512047"} 2023-03-08 11:55:12,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 11:55:12,048 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-08 11:55:12,049 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-08 11:55:12,051 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=964cc659f2b4048d9e6d9a05dea2a708, UNASSIGN}] 2023-03-08 11:55:12,052 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=964cc659f2b4048d9e6d9a05dea2a708, UNASSIGN 2023-03-08 11:55:12,053 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=964cc659f2b4048d9e6d9a05dea2a708, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:12,053 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678276512053"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276512053"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276512053"}]},"ts":"1678276512053"} 2023-03-08 11:55:12,054 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 964cc659f2b4048d9e6d9a05dea2a708, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:12,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 11:55:12,205 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:12,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 964cc659f2b4048d9e6d9a05dea2a708, disabling compactions & flushes 2023-03-08 11:55:12,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:12,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:12,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. after waiting 0 ms 2023-03-08 11:55:12,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:12,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-08 11:55:12,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-08 11:55:12,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708. 2023-03-08 11:55:12,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 964cc659f2b4048d9e6d9a05dea2a708: 2023-03-08 11:55:12,213 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:12,213 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=964cc659f2b4048d9e6d9a05dea2a708, regionState=CLOSED 2023-03-08 11:55:12,213 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678276512213"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276512213"}]},"ts":"1678276512213"} 2023-03-08 11:55:12,216 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-08 11:55:12,216 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 964cc659f2b4048d9e6d9a05dea2a708, server=jenkins-hbase4.apache.org,45551,1678276414946 in 161 msec 2023-03-08 11:55:12,217 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-08 11:55:12,217 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=964cc659f2b4048d9e6d9a05dea2a708, UNASSIGN in 165 msec 2023-03-08 11:55:12,218 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276512218"}]},"ts":"1678276512218"} 2023-03-08 11:55:12,219 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-08 11:55:12,223 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-08 11:55:12,224 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 183 msec 2023-03-08 11:55:12,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 11:55:12,350 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-08 11:55:12,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-03-08 11:55:12,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 11:55:12,362 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 11:55:12,363 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 11:55:12,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-08 11:55:12,366 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:12,368 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1, FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/recovered.edits] 2023-03-08 11:55:12,371 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/1e6b27aadfe74a1c8879eafa80eb5ec4 2023-03-08 11:55:12,372 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/9c4b864881214d3086467dbd644c0119 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/f1/9c4b864881214d3086467dbd644c0119 2023-03-08 11:55:12,374 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/recovered.edits/15.seqid to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708/recovered.edits/15.seqid 2023-03-08 11:55:12,375 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testGetAfterCompaction/964cc659f2b4048d9e6d9a05dea2a708 2023-03-08 11:55:12,375 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-08 11:55:12,375 DEBUG [PEWorker-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-08 11:55:12,376 DEBUG [PEWorker-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-08 11:55:12,378 DEBUG [PEWorker-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308c465b8df418347cd91732945565251e5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308c465b8df418347cd91732945565251e5 2023-03-08 11:55:12,379 DEBUG [PEWorker-5] backup.HFileArchiver(596): Deleted hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-08 11:55:12,381 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 11:55:12,402 WARN [PEWorker-5] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-08 11:55:12,404 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-08 11:55:12,405 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 11:55:12,405 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-08 11:55:12,405 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678276512405"}]},"ts":"9223372036854775807"} 2023-03-08 11:55:12,407 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-08 11:55:12,407 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 964cc659f2b4048d9e6d9a05dea2a708, NAME => 'testGetAfterCompaction,,1678276506706.964cc659f2b4048d9e6d9a05dea2a708.', STARTKEY => '', ENDKEY => ''}] 2023-03-08 11:55:12,407 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-08 11:55:12,407 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1678276512407"}]},"ts":"9223372036854775807"} 2023-03-08 11:55:12,408 INFO [PEWorker-5] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-08 11:55:12,411 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 11:55:12,412 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 56 msec 2023-03-08 11:55:12,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-08 11:55:12,465 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-08 11:55:12,478 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=245 (was 241) - Thread LEAK? -, OpenFileDescriptor=624 (was 622) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=159 (was 164), ProcessCount=170 (was 170), AvailableMemoryMB=7492 (was 7493) 2023-03-08 11:55:12,492 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=245, OpenFileDescriptor=624, MaxFileDescriptor=60000, SystemLoadAverage=159, ProcessCount=170, AvailableMemoryMB=7492 2023-03-08 11:55:12,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 11:55:12,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-08 11:55:12,496 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:55:12,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-08 11:55:12,496 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:55:12,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 11:55:12,498 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:12,498 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:12,498 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606 2023-03-08 11:55:12,499 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4 empty. 2023-03-08 11:55:12,499 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96 empty. 2023-03-08 11:55:12,499 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606 empty. 2023-03-08 11:55:12,499 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:12,500 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606 2023-03-08 11:55:12,500 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:12,500 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-08 11:55:12,540 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 11:55:12,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 11:55:12,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 11:55:12,909 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-08 11:55:12,911 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => 8a2814671292e9b2760956bf02135606, NAME => 'testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:55:12,911 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => 1b2090f00e0a2d838e28358e6d6297e4, NAME => 'testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:55:12,911 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => bb0058b62005e791ee17613797cadc96, NAME => 'testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:55:13,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 11:55:13,322 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:13,322 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing 1b2090f00e0a2d838e28358e6d6297e4, disabling compactions & flushes 2023-03-08 11:55:13,323 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:13,323 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:13,323 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. after waiting 0 ms 2023-03-08 11:55:13,323 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:13,323 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:13,323 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing 8a2814671292e9b2760956bf02135606, disabling compactions & flushes 2023-03-08 11:55:13,324 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. after waiting 0 ms 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:13,324 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing bb0058b62005e791ee17613797cadc96, disabling compactions & flushes 2023-03-08 11:55:13,324 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. after waiting 0 ms 2023-03-08 11:55:13,324 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:13,324 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:13,325 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:13,327 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:55:13,327 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276513327"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276513327"}]},"ts":"1678276513327"} 2023-03-08 11:55:13,328 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276513327"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276513327"}]},"ts":"1678276513327"} 2023-03-08 11:55:13,328 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276513327"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276513327"}]},"ts":"1678276513327"} 2023-03-08 11:55:13,330 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 11:55:13,330 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:55:13,330 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276513330"}]},"ts":"1678276513330"} 2023-03-08 11:55:13,331 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-08 11:55:13,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, ASSIGN}] 2023-03-08 11:55:13,337 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, ASSIGN 2023-03-08 11:55:13,337 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, ASSIGN 2023-03-08 11:55:13,337 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, ASSIGN 2023-03-08 11:55:13,338 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:55:13,338 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:55:13,338 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:55:13,489 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=8a2814671292e9b2760956bf02135606, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:13,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=bb0058b62005e791ee17613797cadc96, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:13,489 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=1b2090f00e0a2d838e28358e6d6297e4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:13,490 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276513489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276513489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276513489"}]},"ts":"1678276513489"} 2023-03-08 11:55:13,490 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276513489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276513489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276513489"}]},"ts":"1678276513489"} 2023-03-08 11:55:13,490 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276513489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276513489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276513489"}]},"ts":"1678276513489"} 2023-03-08 11:55:13,491 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure 8a2814671292e9b2760956bf02135606, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:13,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=80, state=RUNNABLE; OpenRegionProcedure bb0058b62005e791ee17613797cadc96, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:13,494 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=81, state=RUNNABLE; OpenRegionProcedure 1b2090f00e0a2d838e28358e6d6297e4, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:13,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 11:55:13,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:13,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8a2814671292e9b2760956bf02135606, NAME => 'testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 11:55:13,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:13,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:13,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:13,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:13,648 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:13,649 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1 2023-03-08 11:55:13,649 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1 2023-03-08 11:55:13,649 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a2814671292e9b2760956bf02135606 columnFamilyName family1 2023-03-08 11:55:13,650 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(310): Store=8a2814671292e9b2760956bf02135606/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:13,651 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:13,652 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2 2023-03-08 11:55:13,652 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2 2023-03-08 11:55:13,652 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a2814671292e9b2760956bf02135606 columnFamilyName family2 2023-03-08 11:55:13,653 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(310): Store=8a2814671292e9b2760956bf02135606/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:13,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606 2023-03-08 11:55:13,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606 2023-03-08 11:55:13,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:55:13,657 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:13,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:55:13,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8a2814671292e9b2760956bf02135606; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59446008, jitterRate=-0.11418545246124268}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:55:13,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:13,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606., pid=83, masterSystemTime=1678276513643 2023-03-08 11:55:13,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:13,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:13,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:13,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bb0058b62005e791ee17613797cadc96, NAME => 'testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 11:55:13,661 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=8a2814671292e9b2760956bf02135606, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:13,662 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276513661"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276513661"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276513661"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276513661"}]},"ts":"1678276513661"} 2023-03-08 11:55:13,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:13,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:13,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:13,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:13,663 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:13,664 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=82 2023-03-08 11:55:13,664 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure 8a2814671292e9b2760956bf02135606, server=jenkins-hbase4.apache.org,45551,1678276414946 in 172 msec 2023-03-08 11:55:13,664 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1 2023-03-08 11:55:13,664 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1 2023-03-08 11:55:13,665 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb0058b62005e791ee17613797cadc96 columnFamilyName family1 2023-03-08 11:55:13,665 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, ASSIGN in 328 msec 2023-03-08 11:55:13,665 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(310): Store=bb0058b62005e791ee17613797cadc96/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:13,666 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:13,667 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2 2023-03-08 11:55:13,667 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2 2023-03-08 11:55:13,667 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb0058b62005e791ee17613797cadc96 columnFamilyName family2 2023-03-08 11:55:13,668 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(310): Store=bb0058b62005e791ee17613797cadc96/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:13,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:13,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:13,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:55:13,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:13,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:55:13,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bb0058b62005e791ee17613797cadc96; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62866943, jitterRate=-0.06320954859256744}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:55:13,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:13,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96., pid=84, masterSystemTime=1678276513643 2023-03-08 11:55:13,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:13,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:13,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:13,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b2090f00e0a2d838e28358e6d6297e4, NAME => 'testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 11:55:13,676 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=bb0058b62005e791ee17613797cadc96, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:13,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:13,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:13,676 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276513676"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276513676"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276513676"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276513676"}]},"ts":"1678276513676"} 2023-03-08 11:55:13,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:13,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:13,678 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:13,679 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1 2023-03-08 11:55:13,679 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1 2023-03-08 11:55:13,679 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=80 2023-03-08 11:55:13,679 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=80, state=SUCCESS; OpenRegionProcedure bb0058b62005e791ee17613797cadc96, server=jenkins-hbase4.apache.org,45551,1678276414946 in 185 msec 2023-03-08 11:55:13,679 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b2090f00e0a2d838e28358e6d6297e4 columnFamilyName family1 2023-03-08 11:55:13,680 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, ASSIGN in 345 msec 2023-03-08 11:55:13,680 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(310): Store=1b2090f00e0a2d838e28358e6d6297e4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:13,681 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:13,682 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2 2023-03-08 11:55:13,682 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2 2023-03-08 11:55:13,682 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b2090f00e0a2d838e28358e6d6297e4 columnFamilyName family2 2023-03-08 11:55:13,684 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(310): Store=1b2090f00e0a2d838e28358e6d6297e4/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:13,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:13,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:13,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:55:13,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:13,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:55:13,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b2090f00e0a2d838e28358e6d6297e4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60288484, jitterRate=-0.10163158178329468}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:55:13,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:13,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4., pid=85, masterSystemTime=1678276513643 2023-03-08 11:55:13,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:13,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:13,695 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=1b2090f00e0a2d838e28358e6d6297e4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:13,695 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276513695"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276513695"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276513695"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276513695"}]},"ts":"1678276513695"} 2023-03-08 11:55:13,698 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=81 2023-03-08 11:55:13,698 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=81, state=SUCCESS; OpenRegionProcedure 1b2090f00e0a2d838e28358e6d6297e4, server=jenkins-hbase4.apache.org,45551,1678276414946 in 204 msec 2023-03-08 11:55:13,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-03-08 11:55:13,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, ASSIGN in 364 msec 2023-03-08 11:55:13,701 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:55:13,701 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276513701"}]},"ts":"1678276513701"} 2023-03-08 11:55:13,702 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-08 11:55:13,704 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:55:13,705 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2100 sec 2023-03-08 11:55:14,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 11:55:14,602 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-08 11:55:14,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:14,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:14,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:14,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:14,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 11:55:14,636 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 11:55:14,636 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:14,636 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 11:55:14,636 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 11:55:14,636 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:14,636 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:14,639 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:14,639 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:14,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:14,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:14,639 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:14,639 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:14,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:14,640 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:14,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:14,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:14,640 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 11:55:14,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 11:55:14,646 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 11:55:14,646 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:14,646 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 11:55:14,646 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:14,646 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:14,646 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. started... 2023-03-08 11:55:14,646 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. started... 2023-03-08 11:55:14,647 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:55:14,647 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b2090f00e0a2d838e28358e6d6297e4 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 11:55:14,647 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bb0058b62005e791ee17613797cadc96 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 11:55:14,647 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:14,647 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:14,647 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. started... 2023-03-08 11:55:14,648 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 8a2814671292e9b2760956bf02135606 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 11:55:14,676 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308cafe3e79be0c49159be042b6efb707f0 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308cafe3e79be0c49159be042b6efb707f0 2023-03-08 11:55:14,688 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/95af003fa2434171b6ecd72bae8a1c8e 2023-03-08 11:55:15,078 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030871f36923d1904dce9dc63544f1db105f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030871f36923d1904dce9dc63544f1db105f 2023-03-08 11:55:15,089 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303086765cd9fd5364581b2b5cc820219cb0b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303086765cd9fd5364581b2b5cc820219cb0b 2023-03-08 11:55:15,102 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030847ce68641dda433f827e422e602aef3d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030847ce68641dda433f827e422e602aef3d 2023-03-08 11:55:15,484 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/12fb73322f2b462daa9f7f2b5e2177a2 2023-03-08 11:55:15,494 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/3356b1ae0f4c4f8996df9885e552ca20 2023-03-08 11:55:15,507 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/b07367b5aace48be8cff48334d01f5fc 2023-03-08 11:55:15,510 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/95af003fa2434171b6ecd72bae8a1c8e as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/95af003fa2434171b6ecd72bae8a1c8e 2023-03-08 11:55:15,514 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/95af003fa2434171b6ecd72bae8a1c8e, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 11:55:15,514 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/b07367b5aace48be8cff48334d01f5fc as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/b07367b5aace48be8cff48334d01f5fc 2023-03-08 11:55:15,517 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/b07367b5aace48be8cff48334d01f5fc, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 11:55:15,518 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for bb0058b62005e791ee17613797cadc96 in 871ms, sequenceid=5, compaction requested=false 2023-03-08 11:55:15,518 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-08 11:55:15,519 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:15,519 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:15,519 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:55:15,898 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030861f52ec109df4a86b8a622ce66a69748 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030861f52ec109df4a86b8a622ce66a69748 2023-03-08 11:55:15,906 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308054cd3fdc5b14d87a0cea966f45e9c01 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308054cd3fdc5b14d87a0cea966f45e9c01 2023-03-08 11:55:16,303 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/9c92d16d2b9f43529ed6b665e31e6836 2023-03-08 11:55:16,307 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/12fb73322f2b462daa9f7f2b5e2177a2 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/12fb73322f2b462daa9f7f2b5e2177a2 2023-03-08 11:55:16,311 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/12fb73322f2b462daa9f7f2b5e2177a2, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 11:55:16,312 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/9d982a00f7234f3799eff4331f7781ea 2023-03-08 11:55:16,312 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/9c92d16d2b9f43529ed6b665e31e6836 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/9c92d16d2b9f43529ed6b665e31e6836 2023-03-08 11:55:16,315 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/9c92d16d2b9f43529ed6b665e31e6836, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 11:55:16,316 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/3356b1ae0f4c4f8996df9885e552ca20 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/3356b1ae0f4c4f8996df9885e552ca20 2023-03-08 11:55:16,316 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 8a2814671292e9b2760956bf02135606 in 1668ms, sequenceid=5, compaction requested=false 2023-03-08 11:55:16,317 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:16,317 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:16,317 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:55:16,320 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/3356b1ae0f4c4f8996df9885e552ca20, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 11:55:16,321 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/9d982a00f7234f3799eff4331f7781ea as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/9d982a00f7234f3799eff4331f7781ea 2023-03-08 11:55:16,325 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/9d982a00f7234f3799eff4331f7781ea, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 11:55:16,326 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 1b2090f00e0a2d838e28358e6d6297e4 in 1679ms, sequenceid=5, compaction requested=false 2023-03-08 11:55:16,326 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:16,326 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:16,326 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:55:16,326 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:55:16,326 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,326 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 11:55:16,326 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 11:55:16,328 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,328 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:16,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:16,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:16,328 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:16,328 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:16,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:16,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:16,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:16,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:16,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 11:55:16,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e96c947[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:16,329 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 11:55:16,330 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:16,331 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:16,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:16,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:16,331 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 11:55:16,331 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 11:55:16,331 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,331 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:16,331 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:16,332 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 11:55:16,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:16,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:16,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:16,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 11:55:16,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:16,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:16,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:16,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:16,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:16,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:16,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 11:55:16,334 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 11:55:16,334 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:16,334 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:16,334 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 11:55:16,334 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:16,338 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:16,338 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:16,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:16,338 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:16,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:16,338 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:16,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:16,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:16,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:16,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:16,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:16,339 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:16,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:16,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,341 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,341 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:16,341 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:16,342 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:16,342 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:16,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:16,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:16,341 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:16,342 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 11:55:16,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:16,342 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:16,342 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:16,342 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:16,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:16,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:16,342 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:16,342 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:16,442 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:16,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:16,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:16,445 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:16,445 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:16,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:16,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 11:55:16,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 11:55:16,450 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 11:55:16,450 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:16,450 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 11:55:16,450 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 11:55:16,450 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:16,450 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:16,451 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,451 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:16,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:16,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:16,452 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:16,452 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:16,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:16,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:16,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:16,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:16,452 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 11:55:16,453 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 11:55:16,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 11:55:16,453 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 11:55:16,453 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:16,453 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 11:55:16,453 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:16,453 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. started... 2023-03-08 11:55:16,453 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:16,453 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. started... 2023-03-08 11:55:16,453 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b2090f00e0a2d838e28358e6d6297e4 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 11:55:16,453 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bb0058b62005e791ee17613797cadc96 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 11:55:16,456 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:55:16,456 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:16,456 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:16,456 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. started... 2023-03-08 11:55:16,456 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 8a2814671292e9b2760956bf02135606 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 11:55:16,482 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030893ca3401786d4c4bb7ef4ff0c678f2d2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030893ca3401786d4c4bb7ef4ff0c678f2d2 2023-03-08 11:55:16,487 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/26c74f6094ab4fb1a736236655bf5017 2023-03-08 11:55:16,876 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308848093d93c8a439596e91454ff9495f2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308848093d93c8a439596e91454ff9495f2 2023-03-08 11:55:16,882 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303088f1203eb82d14d86b18c22fd2a4819de to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303088f1203eb82d14d86b18c22fd2a4819de 2023-03-08 11:55:16,905 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030808c1a747406644088a4f7542cc0c2930 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030808c1a747406644088a4f7542cc0c2930 2023-03-08 11:55:17,282 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/dbcdd7d7d25445c094214456a69dcade 2023-03-08 11:55:17,286 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/051d4a6cbfe34e7a96dc643c2b20cf3a 2023-03-08 11:55:17,298 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308576a00e3c6064cb79e69ac1ced33974a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308576a00e3c6064cb79e69ac1ced33974a 2023-03-08 11:55:17,309 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/81110e19fe8549e99c21702c394f377b 2023-03-08 11:55:17,313 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/26c74f6094ab4fb1a736236655bf5017 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/26c74f6094ab4fb1a736236655bf5017 2023-03-08 11:55:17,316 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/26c74f6094ab4fb1a736236655bf5017, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 11:55:17,317 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/81110e19fe8549e99c21702c394f377b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/81110e19fe8549e99c21702c394f377b 2023-03-08 11:55:17,322 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/81110e19fe8549e99c21702c394f377b, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 11:55:17,322 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for bb0058b62005e791ee17613797cadc96 in 869ms, sequenceid=9, compaction requested=false 2023-03-08 11:55:17,322 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:17,322 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:17,322 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:55:17,702 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303080f559134c40f47d2b5ae8eb46fbcf448 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303080f559134c40f47d2b5ae8eb46fbcf448 2023-03-08 11:55:17,702 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/b4ac784cd164469a8b3459a552bab0db 2023-03-08 11:55:17,706 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/dbcdd7d7d25445c094214456a69dcade as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/dbcdd7d7d25445c094214456a69dcade 2023-03-08 11:55:17,709 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/dbcdd7d7d25445c094214456a69dcade, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 11:55:17,710 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/b4ac784cd164469a8b3459a552bab0db as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/b4ac784cd164469a8b3459a552bab0db 2023-03-08 11:55:17,713 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:55:17,713 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/b4ac784cd164469a8b3459a552bab0db, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 11:55:17,714 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 1b2090f00e0a2d838e28358e6d6297e4 in 1261ms, sequenceid=9, compaction requested=false 2023-03-08 11:55:17,714 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:17,714 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:17,714 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:55:18,108 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/3a0c4db82b484bbea43a20cc9970f523 2023-03-08 11:55:18,112 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/051d4a6cbfe34e7a96dc643c2b20cf3a as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/051d4a6cbfe34e7a96dc643c2b20cf3a 2023-03-08 11:55:18,115 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/051d4a6cbfe34e7a96dc643c2b20cf3a, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 11:55:18,116 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/3a0c4db82b484bbea43a20cc9970f523 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/3a0c4db82b484bbea43a20cc9970f523 2023-03-08 11:55:18,120 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/3a0c4db82b484bbea43a20cc9970f523, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 11:55:18,120 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 8a2814671292e9b2760956bf02135606 in 1664ms, sequenceid=9, compaction requested=false 2023-03-08 11:55:18,120 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:18,121 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:18,121 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:55:18,121 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:55:18,121 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,121 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 11:55:18,121 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 11:55:18,122 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:18,122 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:18,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:18,123 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:18,123 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:18,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:18,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 11:55:18,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@421a36a7[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:18,124 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 11:55:18,124 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:18,126 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:18,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:18,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:18,127 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 11:55:18,127 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,127 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:18,127 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 11:55:18,127 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:18,128 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 11:55:18,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:18,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:18,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:18,128 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 11:55:18,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:18,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:18,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:18,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:18,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:18,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:18,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 11:55:18,130 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 11:55:18,130 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:18,130 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:18,130 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 11:55:18,130 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:18,132 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:18,132 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:18,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:18,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:18,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:18,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:18,132 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:18,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:18,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:18,132 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:18,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:18,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:18,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:18,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:18,133 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:18,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:18,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:18,135 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:18,135 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,135 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:18,135 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:18,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:18,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 11:55:18,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:18,135 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:18,136 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:18,136 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,136 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:18,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:18,136 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:18,136 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:18,136 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 11:55:18,136 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:18,136 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:18,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:18,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:18,236 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:18,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:18,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:18,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:18,239 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:18,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:18,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 11:55:18,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 11:55:18,243 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 11:55:18,243 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:18,243 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 11:55:18,243 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 11:55:18,243 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:18,243 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:18,245 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,245 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:18,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:18,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:18,245 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:18,245 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:18,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:18,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:18,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:18,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:18,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 11:55:18,246 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 11:55:18,246 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 11:55:18,247 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 11:55:18,247 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:18,247 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 11:55:18,247 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:18,247 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. started... 2023-03-08 11:55:18,247 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:18,247 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:55:18,247 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b2090f00e0a2d838e28358e6d6297e4 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 11:55:18,247 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:18,247 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:18,247 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. started... 2023-03-08 11:55:18,247 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. started... 2023-03-08 11:55:18,247 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bb0058b62005e791ee17613797cadc96 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 11:55:18,247 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 8a2814671292e9b2760956bf02135606 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 11:55:18,270 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303082aec09723e744fe089858f4c25d2aa10 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303082aec09723e744fe089858f4c25d2aa10 2023-03-08 11:55:18,670 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308abdbea5c3a9049b4a8f1ee86acbfe7f0 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308abdbea5c3a9049b4a8f1ee86acbfe7f0 2023-03-08 11:55:18,671 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030877dd088b99974eeb8271403356acfffc to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030877dd088b99974eeb8271403356acfffc 2023-03-08 11:55:18,677 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/27d2cdabb3144f8dad476aebd9d2ba92 2023-03-08 11:55:19,076 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/1cd5f0aa679d4c86b010d1eb4ea4235c 2023-03-08 11:55:19,077 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/bc161e308bab4d49a0b696e98fc1727c 2023-03-08 11:55:19,090 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308d2c5e8e75a784ce1bf17b8c0964b5541 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d2c5e8e75a784ce1bf17b8c0964b5541 2023-03-08 11:55:19,491 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030870c894d1fe9c45bdb1e8bec2c7c3a173 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030870c894d1fe9c45bdb1e8bec2c7c3a173 2023-03-08 11:55:19,491 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030826f1ddad5e534a108c3528f6b6d86c72 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030826f1ddad5e534a108c3528f6b6d86c72 2023-03-08 11:55:19,494 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/450acdbf2fdd4380b22b172baf63edb9 2023-03-08 11:55:19,503 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/27d2cdabb3144f8dad476aebd9d2ba92 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/27d2cdabb3144f8dad476aebd9d2ba92 2023-03-08 11:55:19,506 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/27d2cdabb3144f8dad476aebd9d2ba92, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 11:55:19,507 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/450acdbf2fdd4380b22b172baf63edb9 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/450acdbf2fdd4380b22b172baf63edb9 2023-03-08 11:55:19,510 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/450acdbf2fdd4380b22b172baf63edb9, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 11:55:19,511 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for bb0058b62005e791ee17613797cadc96 in 1264ms, sequenceid=13, compaction requested=true 2023-03-08 11:55:19,511 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:19,511 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:19,511 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:55:19,898 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/3e8179efb3d04e28ab09e2d11966c946 2023-03-08 11:55:19,898 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/855e4339fb014cda9a4a8621eb2c9cc5 2023-03-08 11:55:19,902 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/1cd5f0aa679d4c86b010d1eb4ea4235c as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1cd5f0aa679d4c86b010d1eb4ea4235c 2023-03-08 11:55:19,902 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/bc161e308bab4d49a0b696e98fc1727c as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/bc161e308bab4d49a0b696e98fc1727c 2023-03-08 11:55:19,909 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1cd5f0aa679d4c86b010d1eb4ea4235c, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 11:55:19,910 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/bc161e308bab4d49a0b696e98fc1727c, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 11:55:19,910 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/855e4339fb014cda9a4a8621eb2c9cc5 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/855e4339fb014cda9a4a8621eb2c9cc5 2023-03-08 11:55:19,910 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/3e8179efb3d04e28ab09e2d11966c946 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/3e8179efb3d04e28ab09e2d11966c946 2023-03-08 11:55:19,914 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/855e4339fb014cda9a4a8621eb2c9cc5, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 11:55:19,914 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/3e8179efb3d04e28ab09e2d11966c946, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 11:55:19,915 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 8a2814671292e9b2760956bf02135606 in 1668ms, sequenceid=13, compaction requested=true 2023-03-08 11:55:19,915 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 1b2090f00e0a2d838e28358e6d6297e4 in 1668ms, sequenceid=13, compaction requested=true 2023-03-08 11:55:19,915 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:19,915 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:19,915 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:19,915 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:19,915 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:55:19,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:55:19,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:55:19,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 11:55:19,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 11:55:19,917 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,917 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:19,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:19,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:19,917 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:19,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:19,917 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:19,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:19,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:19,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:19,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 11:55:19,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@769573ba[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:19,919 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 11:55:19,919 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:19,920 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:19,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:19,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:19,920 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 11:55:19,920 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 11:55:19,920 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:19,920 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,920 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:19,921 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,921 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 11:55:19,921 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:19,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:19,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:19,921 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 11:55:19,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:19,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:19,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:19,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:19,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:19,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:19,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 11:55:19,923 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 11:55:19,924 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:19,924 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:19,924 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 11:55:19,924 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:19,928 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:19,928 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:19,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:19,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:19,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:19,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:19,928 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:19,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:19,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:19,929 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:19,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:19,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:19,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:19,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:19,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:19,930 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:19,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:19,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:19,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:19,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:19,932 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:19,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:19,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:19,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:19,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:19,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:19,932 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:19,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 11:55:19,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:19,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:19,932 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 11:55:19,932 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:19,933 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:19,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:19,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:20,033 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:20,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:20,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:20,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:20,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:20,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:20,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 11:55:20,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 11:55:20,039 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 11:55:20,039 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:20,039 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 11:55:20,039 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 11:55:20,040 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:20,040 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:20,041 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:20,041 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:20,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:20,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:20,041 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:20,041 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:20,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:20,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:20,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:20,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:20,042 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 11:55:20,042 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 11:55:20,042 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 11:55:20,042 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 11:55:20,042 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:20,042 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 11:55:20,042 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:20,042 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:20,042 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. started... 2023-03-08 11:55:20,042 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:55:20,042 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:20,042 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. started... 2023-03-08 11:55:20,042 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:20,043 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bb0058b62005e791ee17613797cadc96 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 11:55:20,043 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b2090f00e0a2d838e28358e6d6297e4 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 11:55:20,043 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. started... 2023-03-08 11:55:20,043 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 8a2814671292e9b2760956bf02135606 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 11:55:20,461 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303085e92c651e62347979ce171cd2b619c48 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303085e92c651e62347979ce171cd2b619c48 2023-03-08 11:55:20,461 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303085a93e00ffcfc4680bd8693f36acefb79 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303085a93e00ffcfc4680bd8693f36acefb79 2023-03-08 11:55:20,461 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308c7c09d3168c34702afb2096403f4dbad to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c7c09d3168c34702afb2096403f4dbad 2023-03-08 11:55:20,874 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/e99f9377a8d64fbf9813721496bc314c 2023-03-08 11:55:20,875 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/3194c7487b0945148bb92e7a12c01a06 2023-03-08 11:55:20,875 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/4183389884c8402d912826df5bd5dbfe 2023-03-08 11:55:21,296 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308cef6c2bbc100468792820f193ef34319 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308cef6c2bbc100468792820f193ef34319 2023-03-08 11:55:21,296 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030804b635893f374fd9a728fd22f708ce05 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030804b635893f374fd9a728fd22f708ce05 2023-03-08 11:55:21,296 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030807304ad827064736bd3bbd01e57050d8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030807304ad827064736bd3bbd01e57050d8 2023-03-08 11:55:21,709 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/5908c3bffaf24fcaa736f19e125a250b 2023-03-08 11:55:21,710 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/1f73d40af99a49ff99f80f52c56f4085 2023-03-08 11:55:21,711 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/5a3c6c586c1d4a2da00c3fe70280debf 2023-03-08 11:55:21,715 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/4183389884c8402d912826df5bd5dbfe as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/4183389884c8402d912826df5bd5dbfe 2023-03-08 11:55:21,715 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/3194c7487b0945148bb92e7a12c01a06 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/3194c7487b0945148bb92e7a12c01a06 2023-03-08 11:55:21,716 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/e99f9377a8d64fbf9813721496bc314c as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/e99f9377a8d64fbf9813721496bc314c 2023-03-08 11:55:21,719 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/3194c7487b0945148bb92e7a12c01a06, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 11:55:21,719 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/4183389884c8402d912826df5bd5dbfe, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 11:55:21,720 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/e99f9377a8d64fbf9813721496bc314c, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 11:55:21,720 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/1f73d40af99a49ff99f80f52c56f4085 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/1f73d40af99a49ff99f80f52c56f4085 2023-03-08 11:55:21,720 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/5908c3bffaf24fcaa736f19e125a250b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/5908c3bffaf24fcaa736f19e125a250b 2023-03-08 11:55:21,720 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/5a3c6c586c1d4a2da00c3fe70280debf as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/5a3c6c586c1d4a2da00c3fe70280debf 2023-03-08 11:55:21,724 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/5908c3bffaf24fcaa736f19e125a250b, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 11:55:21,724 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/1f73d40af99a49ff99f80f52c56f4085, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 11:55:21,724 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/5a3c6c586c1d4a2da00c3fe70280debf, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 11:55:21,725 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 1b2090f00e0a2d838e28358e6d6297e4 in 1683ms, sequenceid=17, compaction requested=true 2023-03-08 11:55:21,725 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 8a2814671292e9b2760956bf02135606 in 1682ms, sequenceid=17, compaction requested=true 2023-03-08 11:55:21,725 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:21,725 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for bb0058b62005e791ee17613797cadc96 in 1682ms, sequenceid=17, compaction requested=true 2023-03-08 11:55:21,725 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:21,725 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:21,725 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:21,725 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:55:21,725 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:21,725 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:21,725 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:55:21,725 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:55:21,725 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:55:21,725 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,725 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 11:55:21,726 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 11:55:21,727 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,727 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:21,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:21,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:21,728 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:21,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:21,728 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:21,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:21,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:21,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:21,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 11:55:21,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21e791b6[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:21,729 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 11:55:21,729 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:21,731 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:21,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:21,731 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 11:55:21,731 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 11:55:21,731 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,731 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:21,732 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:21,733 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,733 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 11:55:21,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:21,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:21,733 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:21,733 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 11:55:21,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:21,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:21,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:21,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:21,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:21,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:21,738 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 11:55:21,738 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:21,738 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:21,738 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 11:55:21,739 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 11:55:21,739 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:21,741 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:21,741 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:21,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:21,741 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:21,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:21,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:21,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:21,745 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:21,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:21,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:21,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:21,746 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:21,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:21,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,748 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,748 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:21,748 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:21,748 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:21,748 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:21,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:21,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:21,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 11:55:21,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:21,748 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:21,748 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:21,748 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:21,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:21,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:21,749 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:21,749 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:21,749 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:21,749 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:21,849 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:21,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:21,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 11:55:21,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678276521860 (current time:1678276521860). 2023-03-08 11:55:21,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-08 11:55:21,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] zookeeper.ReadOnlyZKClient(139): Connect 0x16cc1b23 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:55:21,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6f8761c7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:55:21,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:55:21,874 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35384, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:55:21,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x16cc1b23 to 127.0.0.1:51863 2023-03-08 11:55:21,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:55:21,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-08 11:55:21,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 11:55:21,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 11:55:21,885 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 11:55:21,886 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678277121886 2023-03-08 11:55:21,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 11:55:21,886 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1678276521850 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-08 11:55:21,887 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 11:55:21,888 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 11:55:21,888 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-03-08 11:55:21,889 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 11:55:21,889 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1678277121889 2023-03-08 11:55:21,891 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:21,891 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-08 11:55:21,991 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:21,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:21,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 11:55:21,995 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-08 11:55:22,195 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:22,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:22,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 11:55:22,196 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-08 11:55:22,297 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1678276521850 2023-03-08 11:55:22,297 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1678276521850' 2023-03-08 11:55:22,297 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-08 11:55:22,297 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1678276521850' starting 'acquire' 2023-03-08 11:55:22,297 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1678276521850', kicking off acquire phase on members. 2023-03-08 11:55:22,298 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678276521850 2023-03-08 11:55:22,298 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1678276521850 2023-03-08 11:55:22,299 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-08 11:55:22,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-08 11:55:22,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 11:55:22,299 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,299 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:22,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1678276521850 2023-03-08 11:55:22,300 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678276521850 2023-03-08 11:55:22,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-08 11:55:22,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1678276521850 2023-03-08 11:55:22,300 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1678276521850 from table testCompactionWithHFileLink type FLUSH 2023-03-08 11:55:22,302 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1678276521850 2023-03-08 11:55:22,302 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1678276521850' with timeout 300000ms 2023-03-08 11:55:22,302 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-08 11:55:22,302 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1678276521850' starting 'acquire' stage 2023-03-08 11:55:22,303 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1678276521850' locally acquired 2023-03-08 11:55:22,303 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (snaptb-1678276521850) in zk 2023-03-08 11:55:22,304 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1678276521850 2023-03-08 11:55:22,304 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:22,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 11:55:22,304 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678276521850 2023-03-08 11:55:22,304 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1678276521850' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:22,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:22,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:22,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678276521850 2023-03-08 11:55:22,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:22,305 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'snaptb-1678276521850' on coordinator 2023-03-08 11:55:22,305 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1903b83c[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:22,305 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1678276521850' starting 'in-barrier' execution. 2023-03-08 11:55:22,306 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1678276521850 2023-03-08 11:55:22,307 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678276521850 2023-03-08 11:55:22,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1678276521850 2023-03-08 11:55:22,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1678276521850 2023-03-08 11:55:22,308 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1678276521850' received 'reached' from coordinator. 2023-03-08 11:55:22,308 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,308 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:22,309 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:22,309 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-08 11:55:22,309 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:22,309 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:22,309 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-08 11:55:22,309 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. started... 2023-03-08 11:55:22,309 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. started... 2023-03-08 11:55:22,309 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. started... 2023-03-08 11:55:22,309 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:22,309 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:22,309 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:22,310 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.' region-info for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,310 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.' region-info for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,310 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.' region-info for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,472 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-08 11:55:22,472 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:55:22,472 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6fbf5bad to 127.0.0.1:51863 2023-03-08 11:55:22,472 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:55:22,472 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:55:22,472 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3f49ad69 to 127.0.0.1:51863 2023-03-08 11:55:22,473 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:55:22,473 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e2c90ca to 127.0.0.1:51863 2023-03-08 11:55:22,473 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:55:22,473 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:55:22,473 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x715c79f4 to 127.0.0.1:51863 2023-03-08 11:55:22,473 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:55:22,474 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 11:55:22,474 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 11:55:22,474 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 11:55:22,478 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/95af003fa2434171b6ecd72bae8a1c8e, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/26c74f6094ab4fb1a736236655bf5017, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/27d2cdabb3144f8dad476aebd9d2ba92, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/3194c7487b0945148bb92e7a12c01a06] hfiles 2023-03-08 11:55:22,478 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/12fb73322f2b462daa9f7f2b5e2177a2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/051d4a6cbfe34e7a96dc643c2b20cf3a, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1cd5f0aa679d4c86b010d1eb4ea4235c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/e99f9377a8d64fbf9813721496bc314c] hfiles 2023-03-08 11:55:22,478 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/95af003fa2434171b6ecd72bae8a1c8e for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,478 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/12fb73322f2b462daa9f7f2b5e2177a2 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,478 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/3356b1ae0f4c4f8996df9885e552ca20, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/dbcdd7d7d25445c094214456a69dcade, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/bc161e308bab4d49a0b696e98fc1727c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/4183389884c8402d912826df5bd5dbfe] hfiles 2023-03-08 11:55:22,478 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/3356b1ae0f4c4f8996df9885e552ca20 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,481 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/26c74f6094ab4fb1a736236655bf5017 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,481 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/dbcdd7d7d25445c094214456a69dcade for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,481 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/051d4a6cbfe34e7a96dc643c2b20cf3a for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/bc161e308bab4d49a0b696e98fc1727c for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/27d2cdabb3144f8dad476aebd9d2ba92 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/4183389884c8402d912826df5bd5dbfe for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/3194c7487b0945148bb92e7a12c01a06 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1cd5f0aa679d4c86b010d1eb4ea4235c for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/e99f9377a8d64fbf9813721496bc314c for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/9d982a00f7234f3799eff4331f7781ea, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/b4ac784cd164469a8b3459a552bab0db, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/3e8179efb3d04e28ab09e2d11966c946, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/5908c3bffaf24fcaa736f19e125a250b] hfiles 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/9c92d16d2b9f43529ed6b665e31e6836, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/3a0c4db82b484bbea43a20cc9970f523, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/855e4339fb014cda9a4a8621eb2c9cc5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/5a3c6c586c1d4a2da00c3fe70280debf] hfiles 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/9c92d16d2b9f43529ed6b665e31e6836 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/9d982a00f7234f3799eff4331f7781ea for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/b07367b5aace48be8cff48334d01f5fc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/81110e19fe8549e99c21702c394f377b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/450acdbf2fdd4380b22b172baf63edb9, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/1f73d40af99a49ff99f80f52c56f4085] hfiles 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/b4ac784cd164469a8b3459a552bab0db for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/b07367b5aace48be8cff48334d01f5fc for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/3a0c4db82b484bbea43a20cc9970f523 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/81110e19fe8549e99c21702c394f377b for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/855e4339fb014cda9a4a8621eb2c9cc5 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/3e8179efb3d04e28ab09e2d11966c946 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/5a3c6c586c1d4a2da00c3fe70280debf for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/450acdbf2fdd4380b22b172baf63edb9 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/1f73d40af99a49ff99f80f52c56f4085 for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,482 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/5908c3bffaf24fcaa736f19e125a250b for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,497 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:22,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:22,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 11:55:22,498 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-08 11:55:22,889 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. completed. 2023-03-08 11:55:22,889 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. completed. 2023-03-08 11:55:22,889 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:22,889 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:22,889 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. completed. 2023-03-08 11:55:22,889 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:22,890 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-08 11:55:22,890 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-08 11:55:22,890 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-08 11:55:22,890 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-08 11:55:22,890 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,890 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1678276521850' locally completed 2023-03-08 11:55:22,890 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1678276521850' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:22,891 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1678276521850' has notified controller of completion 2023-03-08 11:55:22,891 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,892 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:22,892 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1678276521850' completed. 2023-03-08 11:55:22,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:22,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 11:55:22,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:22,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:22,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678276521850 2023-03-08 11:55:22,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:22,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678276521850 2023-03-08 11:55:22,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1678276521850' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:22,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'snaptb-1678276521850', counting down latch. Waiting for 0 more 2023-03-08 11:55:22,894 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1678276521850' execution completed 2023-03-08 11:55:22,894 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:22,894 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:22,894 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1678276521850 2023-03-08 11:55:22,894 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1678276521850including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-08 11:55:22,897 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678276521850 2023-03-08 11:55:22,897 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678276521850 2023-03-08 11:55:22,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1678276521850 2023-03-08 11:55:22,897 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-08 11:55:22,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678276521850 2023-03-08 11:55:22,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1678276521850 2023-03-08 11:55:22,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:22,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 11:55:22,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-08 11:55:22,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 11:55:22,897 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:22,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678276521850 2023-03-08 11:55:22,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678276521850 2023-03-08 11:55:22,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:22,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678276521850 2023-03-08 11:55:22,898 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:22,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678276521850 2023-03-08 11:55:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,900 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,900 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-08 11:55:22,900 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678276521850 2023-03-08 11:55:22,900 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:22,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-08 11:55:22,900 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-08 11:55:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 11:55:22,900 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1678276521850 2023-03-08 11:55:22,900 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678276521850 2023-03-08 11:55:22,900 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-08 11:55:22,900 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678276521850/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:22,900 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678276521850 2023-03-08 11:55:22,900 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1678276521850 2023-03-08 11:55:22,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-08 11:55:22,900 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-08 11:55:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 11:55:22,900 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678276521850 2023-03-08 11:55:22,901 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678276521850 2023-03-08 11:55:22,902 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303085a93e00ffcfc4680bd8693f36acefb79, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303086765cd9fd5364581b2b5cc820219cb0b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030877dd088b99974eeb8271403356acfffc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308848093d93c8a439596e91454ff9495f2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030871f36923d1904dce9dc63544f1db105f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303088f1203eb82d14d86b18c22fd2a4819de, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308abdbea5c3a9049b4a8f1ee86acbfe7f0, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c7c09d3168c34702afb2096403f4dbad, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303082aec09723e744fe089858f4c25d2aa10, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303085e92c651e62347979ce171cd2b619c48, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030893ca3401786d4c4bb7ef4ff0c678f2d2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308cafe3e79be0c49159be042b6efb707f0] mob files 2023-03-08 11:55:22,902 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303085a93e00ffcfc4680bd8693f36acefb79 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303086765cd9fd5364581b2b5cc820219cb0b 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030877dd088b99974eeb8271403356acfffc 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308848093d93c8a439596e91454ff9495f2 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030871f36923d1904dce9dc63544f1db105f 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303088f1203eb82d14d86b18c22fd2a4819de 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308abdbea5c3a9049b4a8f1ee86acbfe7f0 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c7c09d3168c34702afb2096403f4dbad 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303082aec09723e744fe089858f4c25d2aa10 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303085e92c651e62347979ce171cd2b619c48 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030893ca3401786d4c4bb7ef4ff0c678f2d2 2023-03-08 11:55:22,903 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308cafe3e79be0c49159be042b6efb707f0 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308054cd3fdc5b14d87a0cea966f45e9c01, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308576a00e3c6064cb79e69ac1ced33974a, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030870c894d1fe9c45bdb1e8bec2c7c3a173, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308cef6c2bbc100468792820f193ef34319, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030804b635893f374fd9a728fd22f708ce05, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303080f559134c40f47d2b5ae8eb46fbcf448, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030826f1ddad5e534a108c3528f6b6d86c72, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030861f52ec109df4a86b8a622ce66a69748, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030807304ad827064736bd3bbd01e57050d8, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030808c1a747406644088a4f7542cc0c2930, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030847ce68641dda433f827e422e602aef3d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d2c5e8e75a784ce1bf17b8c0964b5541] mob files 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308054cd3fdc5b14d87a0cea966f45e9c01 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308576a00e3c6064cb79e69ac1ced33974a 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030870c894d1fe9c45bdb1e8bec2c7c3a173 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308cef6c2bbc100468792820f193ef34319 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030804b635893f374fd9a728fd22f708ce05 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303080f559134c40f47d2b5ae8eb46fbcf448 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030826f1ddad5e534a108c3528f6b6d86c72 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030861f52ec109df4a86b8a622ce66a69748 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030807304ad827064736bd3bbd01e57050d8 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030808c1a747406644088a4f7542cc0c2930 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030847ce68641dda433f827e422e602aef3d 2023-03-08 11:55:22,904 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d2c5e8e75a784ce1bf17b8c0964b5541 2023-03-08 11:55:22,998 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:22,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:22,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 11:55:22,999 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-08 11:55:23,309 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1678276521850 2023-03-08 11:55:23,311 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/.tmp/snaptb-1678276521850 2023-03-08 11:55:23,756 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/.tmp/snaptb-1678276521850 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/snaptb-1678276521850 2023-03-08 11:55:23,999 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:24,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:24,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 11:55:24,000 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-03-08 11:55:24,571 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1678276521850 of table testCompactionWithHFileLink completed 2023-03-08 11:55:24,571 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/.tmp/snaptb-1678276521850 2023-03-08 11:55:24,572 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/.tmp/snaptb-1678276521850 2023-03-08 11:55:24,572 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1678276521850 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678276521886Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1678276523309 (+1423 ms)Consolidate snapshot: snaptb-1678276521850 at 1678276523309Loading Region manifests for snaptb-1678276521850 at 1678276523309Writing data manifest for snaptb-1678276521850 at 1678276523321 (+12 ms)Verifying snapshot: snaptb-1678276521850 at 1678276523728 (+407 ms)Snapshot snaptb-1678276521850 of table testCompactionWithHFileLink completed at 1678276524571 (+843 ms) 2023-03-08 11:55:24,573 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 11:55:24,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.6860 sec 2023-03-08 11:55:26,001 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:26,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:26,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-08 11:55:26,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:26,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 11:55:26,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 11:55:26,032 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 11:55:26,032 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:26,032 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 11:55:26,032 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 11:55:26,032 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:26,032 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:26,033 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:26,033 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:26,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:26,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:26,034 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:26,034 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:26,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:26,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:26,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:26,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:26,034 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 11:55:26,035 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 11:55:26,035 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 11:55:26,035 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:26,035 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 11:55:26,035 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:26,035 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. started... 2023-03-08 11:55:26,035 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b2090f00e0a2d838e28358e6d6297e4 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 11:55:26,036 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:26,036 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:55:26,036 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:26,036 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:26,036 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. started... 2023-03-08 11:55:26,036 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. started... 2023-03-08 11:55:26,036 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bb0058b62005e791ee17613797cadc96 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 11:55:26,036 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 8a2814671292e9b2760956bf02135606 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 11:55:26,080 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/5ce7d03aedfd4b2db1be0cb74a2da288 2023-03-08 11:55:26,084 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ce7d03aedfd4b2db1be0cb74a2da288 2023-03-08 11:55:26,473 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/34704e76ce3f48c48b39cdd2b8b3d443 2023-03-08 11:55:26,477 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 34704e76ce3f48c48b39cdd2b8b3d443 2023-03-08 11:55:26,484 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/1e8d8d2a7a254fef9c8dd67518c6f520 2023-03-08 11:55:26,488 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1e8d8d2a7a254fef9c8dd67518c6f520 2023-03-08 11:55:26,502 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/803a50c7befa4921912c26e60d00d9b7 2023-03-08 11:55:26,506 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 803a50c7befa4921912c26e60d00d9b7 2023-03-08 11:55:26,506 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/5ce7d03aedfd4b2db1be0cb74a2da288 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/5ce7d03aedfd4b2db1be0cb74a2da288 2023-03-08 11:55:26,509 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ce7d03aedfd4b2db1be0cb74a2da288 2023-03-08 11:55:26,509 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/5ce7d03aedfd4b2db1be0cb74a2da288, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 11:55:26,510 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/803a50c7befa4921912c26e60d00d9b7 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/803a50c7befa4921912c26e60d00d9b7 2023-03-08 11:55:26,513 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 803a50c7befa4921912c26e60d00d9b7 2023-03-08 11:55:26,513 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/803a50c7befa4921912c26e60d00d9b7, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 11:55:26,514 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 1b2090f00e0a2d838e28358e6d6297e4 in 479ms, sequenceid=24, compaction requested=true 2023-03-08 11:55:26,514 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:26,514 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:26,514 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:55:26,903 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/d05ce7677f4542d5b8e94240f4913e5f 2023-03-08 11:55:26,907 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d05ce7677f4542d5b8e94240f4913e5f 2023-03-08 11:55:26,907 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/1e8d8d2a7a254fef9c8dd67518c6f520 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1e8d8d2a7a254fef9c8dd67518c6f520 2023-03-08 11:55:26,913 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1e8d8d2a7a254fef9c8dd67518c6f520 2023-03-08 11:55:26,913 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1e8d8d2a7a254fef9c8dd67518c6f520, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 11:55:26,913 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/d05ce7677f4542d5b8e94240f4913e5f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/d05ce7677f4542d5b8e94240f4913e5f 2023-03-08 11:55:26,917 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d05ce7677f4542d5b8e94240f4913e5f 2023-03-08 11:55:26,917 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/d05ce7677f4542d5b8e94240f4913e5f, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 11:55:26,918 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 8a2814671292e9b2760956bf02135606 in 882ms, sequenceid=24, compaction requested=true 2023-03-08 11:55:26,918 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:26,918 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:26,918 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:55:27,292 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/07efd6cfbc844f7985c900bc41530199 2023-03-08 11:55:27,295 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 07efd6cfbc844f7985c900bc41530199 2023-03-08 11:55:27,296 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/34704e76ce3f48c48b39cdd2b8b3d443 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/34704e76ce3f48c48b39cdd2b8b3d443 2023-03-08 11:55:27,299 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 34704e76ce3f48c48b39cdd2b8b3d443 2023-03-08 11:55:27,300 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/34704e76ce3f48c48b39cdd2b8b3d443, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 11:55:27,300 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/07efd6cfbc844f7985c900bc41530199 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/07efd6cfbc844f7985c900bc41530199 2023-03-08 11:55:27,304 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 07efd6cfbc844f7985c900bc41530199 2023-03-08 11:55:27,304 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/07efd6cfbc844f7985c900bc41530199, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 11:55:27,305 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for bb0058b62005e791ee17613797cadc96 in 1269ms, sequenceid=24, compaction requested=true 2023-03-08 11:55:27,305 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:27,305 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:55:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:55:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 11:55:27,305 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 11:55:27,306 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,307 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:27,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:27,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:27,307 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:27,307 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:27,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:27,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:27,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:27,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:27,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 11:55:27,308 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 11:55:27,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a3f1ab5[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:27,308 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:27,313 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:27,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:27,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:27,314 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 11:55:27,314 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 11:55:27,314 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:27,314 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,314 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 11:55:27,315 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:27,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:27,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 11:55:27,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:27,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:27,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 11:55:27,317 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 11:55:27,317 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:27,317 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:27,317 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 11:55:27,317 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:27,318 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:27,318 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:27,318 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:27,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:27,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:27,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:27,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:27,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:27,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:27,319 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:27,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:27,320 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:27,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 11:55:27,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:27,322 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:27,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 11:55:27,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 11:55:27,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:27,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 11:55:27,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:27,322 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:27,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:27,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:27,422 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:27,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:27,424 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:27,424 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): 1b2090f00e0a2d838e28358e6d6297e4/family1 is initiating major compaction (all files) 2023-03-08 11:55:27,424 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:27,425 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:27,425 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of 1b2090f00e0a2d838e28358e6d6297e4/family1 in testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:27,425 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/3356b1ae0f4c4f8996df9885e552ca20, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/dbcdd7d7d25445c094214456a69dcade, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/bc161e308bab4d49a0b696e98fc1727c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/4183389884c8402d912826df5bd5dbfe, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/5ce7d03aedfd4b2db1be0cb74a2da288] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp, totalSize=26.7 K 2023-03-08 11:55:27,425 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 3356b1ae0f4c4f8996df9885e552ca20, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514610 2023-03-08 11:55:27,426 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting dbcdd7d7d25445c094214456a69dcade, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516445 2023-03-08 11:55:27,426 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting bc161e308bab4d49a0b696e98fc1727c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518238 2023-03-08 11:55:27,426 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 4183389884c8402d912826df5bd5dbfe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:27,427 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 5ce7d03aedfd4b2db1be0cb74a2da288, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 11:55:27,446 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): 1b2090f00e0a2d838e28358e6d6297e4#family1#compaction#106 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:27,854 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308c88ec3b0499346da9feab61b9b1b4bd9_del 2023-03-08 11:55:27,854 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308c88ec3b0499346da9feab61b9b1b4bd9_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308c88ec3b0499346da9feab61b9b1b4bd9_del 2023-03-08 11:55:28,667 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 747754bd5443434fa3da4893b56ea646 2023-03-08 11:55:28,668 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/747754bd5443434fa3da4893b56ea646 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/747754bd5443434fa3da4893b56ea646 2023-03-08 11:55:28,671 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 747754bd5443434fa3da4893b56ea646 2023-03-08 11:55:28,672 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1b2090f00e0a2d838e28358e6d6297e4/family1 of 1b2090f00e0a2d838e28358e6d6297e4 into 747754bd5443434fa3da4893b56ea646(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:28,672 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:28,672 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:28,673 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): 1b2090f00e0a2d838e28358e6d6297e4/family2 is initiating major compaction (all files) 2023-03-08 11:55:28,673 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:28,673 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:28,673 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of 1b2090f00e0a2d838e28358e6d6297e4/family2 in testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:28,673 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/9d982a00f7234f3799eff4331f7781ea, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/b4ac784cd164469a8b3459a552bab0db, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/3e8179efb3d04e28ab09e2d11966c946, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/5908c3bffaf24fcaa736f19e125a250b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/803a50c7befa4921912c26e60d00d9b7] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp, totalSize=25.4 K 2023-03-08 11:55:28,673 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 9d982a00f7234f3799eff4331f7781ea, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514610 2023-03-08 11:55:28,674 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting b4ac784cd164469a8b3459a552bab0db, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516445 2023-03-08 11:55:28,674 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 3e8179efb3d04e28ab09e2d11966c946, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518238 2023-03-08 11:55:28,674 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 5908c3bffaf24fcaa736f19e125a250b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:28,675 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 803a50c7befa4921912c26e60d00d9b7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 11:55:28,680 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/3356b1ae0f4c4f8996df9885e552ca20, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/dbcdd7d7d25445c094214456a69dcade, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/bc161e308bab4d49a0b696e98fc1727c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/4183389884c8402d912826df5bd5dbfe, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/5ce7d03aedfd4b2db1be0cb74a2da288] to archive 2023-03-08 11:55:28,681 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:28,683 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/3356b1ae0f4c4f8996df9885e552ca20 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/3356b1ae0f4c4f8996df9885e552ca20 2023-03-08 11:55:28,684 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/dbcdd7d7d25445c094214456a69dcade to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/dbcdd7d7d25445c094214456a69dcade 2023-03-08 11:55:28,685 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/bc161e308bab4d49a0b696e98fc1727c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/bc161e308bab4d49a0b696e98fc1727c 2023-03-08 11:55:28,687 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/4183389884c8402d912826df5bd5dbfe to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/4183389884c8402d912826df5bd5dbfe 2023-03-08 11:55:28,687 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): 1b2090f00e0a2d838e28358e6d6297e4#family2#compaction#107 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:28,687 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/5ce7d03aedfd4b2db1be0cb74a2da288 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/5ce7d03aedfd4b2db1be0cb74a2da288 2023-03-08 11:55:29,096 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308dd803c8ec5b24151bcdb8b045013a73b_del 2023-03-08 11:55:29,096 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308dd803c8ec5b24151bcdb8b045013a73b_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308dd803c8ec5b24151bcdb8b045013a73b_del 2023-03-08 11:55:29,909 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0faf115824b3420db547c12da218e57f 2023-03-08 11:55:29,909 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/0faf115824b3420db547c12da218e57f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/0faf115824b3420db547c12da218e57f 2023-03-08 11:55:29,912 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0faf115824b3420db547c12da218e57f 2023-03-08 11:55:29,913 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1b2090f00e0a2d838e28358e6d6297e4/family2 of 1b2090f00e0a2d838e28358e6d6297e4 into 0faf115824b3420db547c12da218e57f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:29,913 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:29,914 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:29,914 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): bb0058b62005e791ee17613797cadc96/family1 is initiating major compaction (all files) 2023-03-08 11:55:29,914 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:29,914 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:29,914 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of bb0058b62005e791ee17613797cadc96/family1 in testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:29,914 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/95af003fa2434171b6ecd72bae8a1c8e, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/26c74f6094ab4fb1a736236655bf5017, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/27d2cdabb3144f8dad476aebd9d2ba92, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/3194c7487b0945148bb92e7a12c01a06, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/34704e76ce3f48c48b39cdd2b8b3d443] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp, totalSize=26.7 K 2023-03-08 11:55:29,914 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 95af003fa2434171b6ecd72bae8a1c8e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514609 2023-03-08 11:55:29,915 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 26c74f6094ab4fb1a736236655bf5017, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516444 2023-03-08 11:55:29,915 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 27d2cdabb3144f8dad476aebd9d2ba92, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518238 2023-03-08 11:55:29,915 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 3194c7487b0945148bb92e7a12c01a06, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:29,916 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 34704e76ce3f48c48b39cdd2b8b3d443, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 11:55:29,925 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): bb0058b62005e791ee17613797cadc96#family1#compaction#108 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:29,937 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308c6ba2831d3f84555b35012a986487860_del 2023-03-08 11:55:29,937 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308c6ba2831d3f84555b35012a986487860_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308c6ba2831d3f84555b35012a986487860_del 2023-03-08 11:55:29,972 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/9d982a00f7234f3799eff4331f7781ea, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/b4ac784cd164469a8b3459a552bab0db, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/3e8179efb3d04e28ab09e2d11966c946, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/5908c3bffaf24fcaa736f19e125a250b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/803a50c7befa4921912c26e60d00d9b7] to archive 2023-03-08 11:55:29,973 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:29,974 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/9d982a00f7234f3799eff4331f7781ea to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/9d982a00f7234f3799eff4331f7781ea 2023-03-08 11:55:29,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/b4ac784cd164469a8b3459a552bab0db to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/b4ac784cd164469a8b3459a552bab0db 2023-03-08 11:55:29,976 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/3e8179efb3d04e28ab09e2d11966c946 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/3e8179efb3d04e28ab09e2d11966c946 2023-03-08 11:55:29,977 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/5908c3bffaf24fcaa736f19e125a250b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/5908c3bffaf24fcaa736f19e125a250b 2023-03-08 11:55:29,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/803a50c7befa4921912c26e60d00d9b7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/803a50c7befa4921912c26e60d00d9b7 2023-03-08 11:55:30,754 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c65ac81cb05c4576917d2555d97a3d4a 2023-03-08 11:55:30,754 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/c65ac81cb05c4576917d2555d97a3d4a as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/c65ac81cb05c4576917d2555d97a3d4a 2023-03-08 11:55:30,758 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c65ac81cb05c4576917d2555d97a3d4a 2023-03-08 11:55:30,759 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bb0058b62005e791ee17613797cadc96/family1 of bb0058b62005e791ee17613797cadc96 into c65ac81cb05c4576917d2555d97a3d4a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:55:30,759 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:30,759 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:30,759 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): bb0058b62005e791ee17613797cadc96/family2 is initiating major compaction (all files) 2023-03-08 11:55:30,759 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:30,759 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:30,760 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of bb0058b62005e791ee17613797cadc96/family2 in testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:30,760 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/b07367b5aace48be8cff48334d01f5fc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/81110e19fe8549e99c21702c394f377b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/450acdbf2fdd4380b22b172baf63edb9, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/1f73d40af99a49ff99f80f52c56f4085, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/07efd6cfbc844f7985c900bc41530199] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp, totalSize=25.4 K 2023-03-08 11:55:30,760 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting b07367b5aace48be8cff48334d01f5fc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514609 2023-03-08 11:55:30,760 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 81110e19fe8549e99c21702c394f377b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516444 2023-03-08 11:55:30,761 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 450acdbf2fdd4380b22b172baf63edb9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518238 2023-03-08 11:55:30,761 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 1f73d40af99a49ff99f80f52c56f4085, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:30,761 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 07efd6cfbc844f7985c900bc41530199, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 11:55:30,772 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): bb0058b62005e791ee17613797cadc96#family2#compaction#109 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:30,772 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/95af003fa2434171b6ecd72bae8a1c8e, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/26c74f6094ab4fb1a736236655bf5017, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/27d2cdabb3144f8dad476aebd9d2ba92, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/3194c7487b0945148bb92e7a12c01a06, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/34704e76ce3f48c48b39cdd2b8b3d443] to archive 2023-03-08 11:55:30,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:30,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/95af003fa2434171b6ecd72bae8a1c8e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/95af003fa2434171b6ecd72bae8a1c8e 2023-03-08 11:55:30,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/26c74f6094ab4fb1a736236655bf5017 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/26c74f6094ab4fb1a736236655bf5017 2023-03-08 11:55:30,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/27d2cdabb3144f8dad476aebd9d2ba92 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/27d2cdabb3144f8dad476aebd9d2ba92 2023-03-08 11:55:30,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/3194c7487b0945148bb92e7a12c01a06 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/3194c7487b0945148bb92e7a12c01a06 2023-03-08 11:55:30,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/34704e76ce3f48c48b39cdd2b8b3d443 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/34704e76ce3f48c48b39cdd2b8b3d443 2023-03-08 11:55:31,180 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308a5b12cdd84a64eb280792a364472ded7_del 2023-03-08 11:55:31,181 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308a5b12cdd84a64eb280792a364472ded7_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308a5b12cdd84a64eb280792a364472ded7_del 2023-03-08 11:55:31,593 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc79ed68bbcc4211a348d8730b95b905 2023-03-08 11:55:31,594 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/fc79ed68bbcc4211a348d8730b95b905 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/fc79ed68bbcc4211a348d8730b95b905 2023-03-08 11:55:31,597 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc79ed68bbcc4211a348d8730b95b905 2023-03-08 11:55:31,598 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bb0058b62005e791ee17613797cadc96/family2 of bb0058b62005e791ee17613797cadc96 into fc79ed68bbcc4211a348d8730b95b905(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:55:31,598 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:31,598 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:31,598 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): 8a2814671292e9b2760956bf02135606/family1 is initiating major compaction (all files) 2023-03-08 11:55:31,598 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:31,598 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:31,598 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of 8a2814671292e9b2760956bf02135606/family1 in testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:31,599 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/12fb73322f2b462daa9f7f2b5e2177a2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/051d4a6cbfe34e7a96dc643c2b20cf3a, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1cd5f0aa679d4c86b010d1eb4ea4235c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/e99f9377a8d64fbf9813721496bc314c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1e8d8d2a7a254fef9c8dd67518c6f520] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp, totalSize=26.7 K 2023-03-08 11:55:31,599 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 12fb73322f2b462daa9f7f2b5e2177a2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514611 2023-03-08 11:55:31,599 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 051d4a6cbfe34e7a96dc643c2b20cf3a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516445 2023-03-08 11:55:31,599 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 1cd5f0aa679d4c86b010d1eb4ea4235c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518239 2023-03-08 11:55:31,600 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting e99f9377a8d64fbf9813721496bc314c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:31,600 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 1e8d8d2a7a254fef9c8dd67518c6f520, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 11:55:31,613 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): 8a2814671292e9b2760956bf02135606#family1#compaction#110 average throughput is 0.28 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:31,620 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308605300895d584f728f55409ac29d7208_del 2023-03-08 11:55:31,620 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308605300895d584f728f55409ac29d7208_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308605300895d584f728f55409ac29d7208_del 2023-03-08 11:55:31,672 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/b07367b5aace48be8cff48334d01f5fc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/81110e19fe8549e99c21702c394f377b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/450acdbf2fdd4380b22b172baf63edb9, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/1f73d40af99a49ff99f80f52c56f4085, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/07efd6cfbc844f7985c900bc41530199] to archive 2023-03-08 11:55:31,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:31,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/b07367b5aace48be8cff48334d01f5fc to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/b07367b5aace48be8cff48334d01f5fc 2023-03-08 11:55:31,675 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/81110e19fe8549e99c21702c394f377b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/81110e19fe8549e99c21702c394f377b 2023-03-08 11:55:31,676 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/450acdbf2fdd4380b22b172baf63edb9 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/450acdbf2fdd4380b22b172baf63edb9 2023-03-08 11:55:31,677 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/1f73d40af99a49ff99f80f52c56f4085 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/1f73d40af99a49ff99f80f52c56f4085 2023-03-08 11:55:31,678 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/07efd6cfbc844f7985c900bc41530199 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/07efd6cfbc844f7985c900bc41530199 2023-03-08 11:55:32,435 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 34dc99c54d59489f8d38971f6dcdc3cc 2023-03-08 11:55:32,435 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/34dc99c54d59489f8d38971f6dcdc3cc as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/34dc99c54d59489f8d38971f6dcdc3cc 2023-03-08 11:55:32,439 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 34dc99c54d59489f8d38971f6dcdc3cc 2023-03-08 11:55:32,440 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8a2814671292e9b2760956bf02135606/family1 of 8a2814671292e9b2760956bf02135606 into 34dc99c54d59489f8d38971f6dcdc3cc(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:55:32,440 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:32,440 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:32,440 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): 8a2814671292e9b2760956bf02135606/family2 is initiating major compaction (all files) 2023-03-08 11:55:32,441 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:32,441 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:32,442 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of 8a2814671292e9b2760956bf02135606/family2 in testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:32,442 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/9c92d16d2b9f43529ed6b665e31e6836, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/3a0c4db82b484bbea43a20cc9970f523, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/855e4339fb014cda9a4a8621eb2c9cc5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/5a3c6c586c1d4a2da00c3fe70280debf, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/d05ce7677f4542d5b8e94240f4913e5f] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp, totalSize=25.4 K 2023-03-08 11:55:32,442 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 9c92d16d2b9f43529ed6b665e31e6836, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514611 2023-03-08 11:55:32,442 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 3a0c4db82b484bbea43a20cc9970f523, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516445 2023-03-08 11:55:32,443 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 855e4339fb014cda9a4a8621eb2c9cc5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518239 2023-03-08 11:55:32,443 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 5a3c6c586c1d4a2da00c3fe70280debf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:32,443 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting d05ce7677f4542d5b8e94240f4913e5f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 11:55:32,452 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): 8a2814671292e9b2760956bf02135606#family2#compaction#111 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:32,472 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/12fb73322f2b462daa9f7f2b5e2177a2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/051d4a6cbfe34e7a96dc643c2b20cf3a, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1cd5f0aa679d4c86b010d1eb4ea4235c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/e99f9377a8d64fbf9813721496bc314c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1e8d8d2a7a254fef9c8dd67518c6f520] to archive 2023-03-08 11:55:32,473 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:32,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/12fb73322f2b462daa9f7f2b5e2177a2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/12fb73322f2b462daa9f7f2b5e2177a2 2023-03-08 11:55:32,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/051d4a6cbfe34e7a96dc643c2b20cf3a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/051d4a6cbfe34e7a96dc643c2b20cf3a 2023-03-08 11:55:32,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1cd5f0aa679d4c86b010d1eb4ea4235c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1cd5f0aa679d4c86b010d1eb4ea4235c 2023-03-08 11:55:32,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/e99f9377a8d64fbf9813721496bc314c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/e99f9377a8d64fbf9813721496bc314c 2023-03-08 11:55:32,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1e8d8d2a7a254fef9c8dd67518c6f520 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/1e8d8d2a7a254fef9c8dd67518c6f520 2023-03-08 11:55:32,860 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303085a7dc8698c054ac382e595c8fd20426b_del 2023-03-08 11:55:32,861 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303085a7dc8698c054ac382e595c8fd20426b_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085a7dc8698c054ac382e595c8fd20426b_del 2023-03-08 11:55:33,674 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b9d569129c7c4e7496505f062634856a 2023-03-08 11:55:33,675 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/b9d569129c7c4e7496505f062634856a as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/b9d569129c7c4e7496505f062634856a 2023-03-08 11:55:33,678 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b9d569129c7c4e7496505f062634856a 2023-03-08 11:55:33,679 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8a2814671292e9b2760956bf02135606/family2 of 8a2814671292e9b2760956bf02135606 into b9d569129c7c4e7496505f062634856a(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:33,679 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:33,684 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:33,687 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:33,691 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:33,693 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:33,699 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 11:55:33,730 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-08 11:55:33,732 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308c6ba2831d3f84555b35012a986487860_del 2023-03-08 11:55:33,735 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308c88ec3b0499346da9feab61b9b1b4bd9_del 2023-03-08 11:55:33,738 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308605300895d584f728f55409ac29d7208_del 2023-03-08 11:55:33,738 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:33,738 DEBUG [Listener at localhost/36211] zookeeper.ReadOnlyZKClient(139): Connect 0x7c526350 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:55:33,745 DEBUG [Listener at localhost/36211] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@20eae3e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:55:33,745 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:33,746 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:33,750 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:33,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/9c92d16d2b9f43529ed6b665e31e6836, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/3a0c4db82b484bbea43a20cc9970f523, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/855e4339fb014cda9a4a8621eb2c9cc5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/5a3c6c586c1d4a2da00c3fe70280debf, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/d05ce7677f4542d5b8e94240f4913e5f] to archive 2023-03-08 11:55:33,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:33,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/9c92d16d2b9f43529ed6b665e31e6836 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/9c92d16d2b9f43529ed6b665e31e6836 2023-03-08 11:55:33,782 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/3a0c4db82b484bbea43a20cc9970f523 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/3a0c4db82b484bbea43a20cc9970f523 2023-03-08 11:55:33,783 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/855e4339fb014cda9a4a8621eb2c9cc5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/855e4339fb014cda9a4a8621eb2c9cc5 2023-03-08 11:55:33,787 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/5a3c6c586c1d4a2da00c3fe70280debf to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/5a3c6c586c1d4a2da00c3fe70280debf 2023-03-08 11:55:33,790 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/d05ce7677f4542d5b8e94240f4913e5f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/d05ce7677f4542d5b8e94240f4913e5f 2023-03-08 11:55:34,233 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030834a03b91133e4394b759eec8934b6b90 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030834a03b91133e4394b759eec8934b6b90 2023-03-08 11:55:34,234 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:34,235 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308cd60b94817224f84b309293e4ed30176 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308cd60b94817224f84b309293e4ed30176 2023-03-08 11:55:34,235 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:34,251 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:55:34,255 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47224, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:55:34,260 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:55:34,262 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60216, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:55:34,272 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/b82a571a98ef4520b4446376ac27450a first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-08 11:55:34,272 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/1065218564db4a3582ad265bb6777ddf first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-08 11:55:34,274 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/b82a571a98ef4520b4446376ac27450a}] 2023-03-08 11:55:34,274 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/1065218564db4a3582ad265bb6777ddf}] 2023-03-08 11:55:34,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/1065218564db4a3582ad265bb6777ddf for inclusion in 8a2814671292e9b2760956bf02135606/family1 2023-03-08 11:55:34,279 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/b82a571a98ef4520b4446376ac27450a for inclusion in 1b2090f00e0a2d838e28358e6d6297e4/family1 2023-03-08 11:55:34,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-08 11:55:34,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 11:55:34,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-08 11:55:34,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 11:55:34,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:34,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:34,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/b82a571a98ef4520b4446376ac27450a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__9uqj0cb3npk2835kjthokej6kqohinlf08558dvjagk0qv0p60ehcnaoeov69smo/family1/b82a571a98ef4520b4446376ac27450a 2023-03-08 11:55:34,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/1065218564db4a3582ad265bb6777ddf to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__4bglecg6l4rn2ff89fi37rchlcv9elnvfdpo0j32sib7t4h4m9ulivgagd137tqs/family1/1065218564db4a3582ad265bb6777ddf 2023-03-08 11:55:34,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__9uqj0cb3npk2835kjthokej6kqohinlf08558dvjagk0qv0p60ehcnaoeov69smo/family1/b82a571a98ef4520b4446376ac27450a as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/9393d516f2c941999c98dd13e993e44b_SeqId_29_ 2023-03-08 11:55:34,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__4bglecg6l4rn2ff89fi37rchlcv9elnvfdpo0j32sib7t4h4m9ulivgagd137tqs/family1/1065218564db4a3582ad265bb6777ddf as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/f9a342fca44c47d8a8766eecb8e40e68_SeqId_29_ 2023-03-08 11:55:34,284 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__9uqj0cb3npk2835kjthokej6kqohinlf08558dvjagk0qv0p60ehcnaoeov69smo/family1/b82a571a98ef4520b4446376ac27450a into 1b2090f00e0a2d838e28358e6d6297e4/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/9393d516f2c941999c98dd13e993e44b_SeqId_29_ - updating store file list. 2023-03-08 11:55:34,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__4bglecg6l4rn2ff89fi37rchlcv9elnvfdpo0j32sib7t4h4m9ulivgagd137tqs/family1/1065218564db4a3582ad265bb6777ddf into 8a2814671292e9b2760956bf02135606/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/f9a342fca44c47d8a8766eecb8e40e68_SeqId_29_ - updating store file list. 2023-03-08 11:55:34,287 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/9393d516f2c941999c98dd13e993e44b_SeqId_29_ into 1b2090f00e0a2d838e28358e6d6297e4/family1 2023-03-08 11:55:34,287 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__9uqj0cb3npk2835kjthokej6kqohinlf08558dvjagk0qv0p60ehcnaoeov69smo/family1/b82a571a98ef4520b4446376ac27450a into 1b2090f00e0a2d838e28358e6d6297e4/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/9393d516f2c941999c98dd13e993e44b_SeqId_29_) 2023-03-08 11:55:34,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__9uqj0cb3npk2835kjthokej6kqohinlf08558dvjagk0qv0p60ehcnaoeov69smo/family1/b82a571a98ef4520b4446376ac27450a 2023-03-08 11:55:34,289 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:34,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/f9a342fca44c47d8a8766eecb8e40e68_SeqId_29_ into 8a2814671292e9b2760956bf02135606/family1 2023-03-08 11:55:34,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__4bglecg6l4rn2ff89fi37rchlcv9elnvfdpo0j32sib7t4h4m9ulivgagd137tqs/family1/1065218564db4a3582ad265bb6777ddf into 8a2814671292e9b2760956bf02135606/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/f9a342fca44c47d8a8766eecb8e40e68_SeqId_29_) 2023-03-08 11:55:34,290 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:34,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__4bglecg6l4rn2ff89fi37rchlcv9elnvfdpo0j32sib7t4h4m9ulivgagd137tqs/family1/1065218564db4a3582ad265bb6777ddf 2023-03-08 11:55:34,291 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303085a93e00ffcfc4680bd8693f36acefb79 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303085a93e00ffcfc4680bd8693f36acefb79 2023-03-08 11:55:34,292 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:34,292 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:34,292 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303086765cd9fd5364581b2b5cc820219cb0b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303086765cd9fd5364581b2b5cc820219cb0b 2023-03-08 11:55:34,293 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030877dd088b99974eeb8271403356acfffc to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030877dd088b99974eeb8271403356acfffc 2023-03-08 11:55:34,293 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030871f36923d1904dce9dc63544f1db105f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030871f36923d1904dce9dc63544f1db105f 2023-03-08 11:55:34,294 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308848093d93c8a439596e91454ff9495f2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308848093d93c8a439596e91454ff9495f2 2023-03-08 11:55:34,294 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 11:55:34,294 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303088f1203eb82d14d86b18c22fd2a4819de to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303088f1203eb82d14d86b18c22fd2a4819de 2023-03-08 11:55:34,294 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 11:55:34,295 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:34,295 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308abdbea5c3a9049b4a8f1ee86acbfe7f0 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308abdbea5c3a9049b4a8f1ee86acbfe7f0 2023-03-08 11:55:34,296 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c7c09d3168c34702afb2096403f4dbad to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c7c09d3168c34702afb2096403f4dbad 2023-03-08 11:55:34,296 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 11:55:34,296 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 11:55:34,297 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:34,628 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303082f6ec3af72ad4470917b6c949c92ea92 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303082f6ec3af72ad4470917b6c949c92ea92 2023-03-08 11:55:34,629 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:34,651 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/72e2d5e5992b44fbbdbed02dd409eac7 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-08 11:55:34,651 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/72e2d5e5992b44fbbdbed02dd409eac7}] 2023-03-08 11:55:34,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/72e2d5e5992b44fbbdbed02dd409eac7 for inclusion in bb0058b62005e791ee17613797cadc96/family1 2023-03-08 11:55:34,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-08 11:55:34,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 11:55:34,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:34,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/72e2d5e5992b44fbbdbed02dd409eac7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__pmal4eh22a393bc7v8a34473ggidpf2g64iah52dv5k3t0s4perbhp6fs979rkgj/family1/72e2d5e5992b44fbbdbed02dd409eac7 2023-03-08 11:55:34,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__pmal4eh22a393bc7v8a34473ggidpf2g64iah52dv5k3t0s4perbhp6fs979rkgj/family1/72e2d5e5992b44fbbdbed02dd409eac7 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/630e1c12044e45edb608dfe99d90d76a_SeqId_29_ 2023-03-08 11:55:34,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__pmal4eh22a393bc7v8a34473ggidpf2g64iah52dv5k3t0s4perbhp6fs979rkgj/family1/72e2d5e5992b44fbbdbed02dd409eac7 into bb0058b62005e791ee17613797cadc96/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/630e1c12044e45edb608dfe99d90d76a_SeqId_29_ - updating store file list. 2023-03-08 11:55:34,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/630e1c12044e45edb608dfe99d90d76a_SeqId_29_ into bb0058b62005e791ee17613797cadc96/family1 2023-03-08 11:55:34,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__pmal4eh22a393bc7v8a34473ggidpf2g64iah52dv5k3t0s4perbhp6fs979rkgj/family1/72e2d5e5992b44fbbdbed02dd409eac7 into bb0058b62005e791ee17613797cadc96/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/630e1c12044e45edb608dfe99d90d76a_SeqId_29_) 2023-03-08 11:55:34,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__pmal4eh22a393bc7v8a34473ggidpf2g64iah52dv5k3t0s4perbhp6fs979rkgj/family1/72e2d5e5992b44fbbdbed02dd409eac7 2023-03-08 11:55:34,665 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:34,665 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:34,666 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303082aec09723e744fe089858f4c25d2aa10 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303082aec09723e744fe089858f4c25d2aa10 2023-03-08 11:55:34,667 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303085e92c651e62347979ce171cd2b619c48 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303085e92c651e62347979ce171cd2b619c48 2023-03-08 11:55:34,668 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030893ca3401786d4c4bb7ef4ff0c678f2d2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030893ca3401786d4c4bb7ef4ff0c678f2d2 2023-03-08 11:55:34,669 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308cafe3e79be0c49159be042b6efb707f0 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308cafe3e79be0c49159be042b6efb707f0 2023-03-08 11:55:34,669 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 11:55:34,669 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 11:55:34,669 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:34,669 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:34,669 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:34,669 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308c6ba2831d3f84555b35012a986487860_del] 2023-03-08 11:55:34,670 DEBUG [Listener at localhost/36211] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:34,671 DEBUG [Listener at localhost/36211] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308c6ba2831d3f84555b35012a986487860_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308c6ba2831d3f84555b35012a986487860_del 2023-03-08 11:55:34,671 INFO [Listener at localhost/36211] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 11:55:34,671 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308c88ec3b0499346da9feab61b9b1b4bd9_del] 2023-03-08 11:55:34,671 DEBUG [Listener at localhost/36211] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:34,673 DEBUG [Listener at localhost/36211] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308c88ec3b0499346da9feab61b9b1b4bd9_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308c88ec3b0499346da9feab61b9b1b4bd9_del 2023-03-08 11:55:34,673 INFO [Listener at localhost/36211] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 11:55:34,673 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308605300895d584f728f55409ac29d7208_del] 2023-03-08 11:55:34,674 DEBUG [Listener at localhost/36211] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:34,675 DEBUG [Listener at localhost/36211] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308605300895d584f728f55409ac29d7208_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308605300895d584f728f55409ac29d7208_del 2023-03-08 11:55:34,675 INFO [Listener at localhost/36211] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 11:55:34,677 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:34,679 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:34,682 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:34,683 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:34,689 INFO [Listener at localhost/36211] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-08 11:55:34,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-03-08 11:55:34,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-08 11:55:34,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 11:55:34,692 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276534692"}]},"ts":"1678276534692"} 2023-03-08 11:55:34,694 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-08 11:55:34,695 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-08 11:55:34,696 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, UNASSIGN}] 2023-03-08 11:55:34,697 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, UNASSIGN 2023-03-08 11:55:34,697 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, UNASSIGN 2023-03-08 11:55:34,697 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, UNASSIGN 2023-03-08 11:55:34,698 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=8a2814671292e9b2760956bf02135606, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:34,698 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=bb0058b62005e791ee17613797cadc96, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:34,698 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276534698"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276534698"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276534698"}]},"ts":"1678276534698"} 2023-03-08 11:55:34,698 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=1b2090f00e0a2d838e28358e6d6297e4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:34,698 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276534698"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276534698"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276534698"}]},"ts":"1678276534698"} 2023-03-08 11:55:34,698 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276534698"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276534698"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276534698"}]},"ts":"1678276534698"} 2023-03-08 11:55:34,699 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure 8a2814671292e9b2760956bf02135606, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:34,700 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure bb0058b62005e791ee17613797cadc96, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:34,700 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=90, state=RUNNABLE; CloseRegionProcedure 1b2090f00e0a2d838e28358e6d6297e4, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:34,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 11:55:34,851 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:34,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bb0058b62005e791ee17613797cadc96, disabling compactions & flushes 2023-03-08 11:55:34,851 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:34,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:34,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. after waiting 0 ms 2023-03-08 11:55:34,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:34,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 11:55:34,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:34,856 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:34,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:34,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:34,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b2090f00e0a2d838e28358e6d6297e4, disabling compactions & flushes 2023-03-08 11:55:34,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:34,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:34,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. after waiting 0 ms 2023-03-08 11:55:34,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:34,858 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=bb0058b62005e791ee17613797cadc96, regionState=CLOSED 2023-03-08 11:55:34,858 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276534858"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276534858"}]},"ts":"1678276534858"} 2023-03-08 11:55:34,861 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=89 2023-03-08 11:55:34,861 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure bb0058b62005e791ee17613797cadc96, server=jenkins-hbase4.apache.org,45551,1678276414946 in 159 msec 2023-03-08 11:55:34,861 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 11:55:34,862 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, UNASSIGN in 165 msec 2023-03-08 11:55:34,862 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:34,862 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:34,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:34,864 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:34,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8a2814671292e9b2760956bf02135606, disabling compactions & flushes 2023-03-08 11:55:34,864 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:34,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:34,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. after waiting 0 ms 2023-03-08 11:55:34,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:34,864 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=1b2090f00e0a2d838e28358e6d6297e4, regionState=CLOSED 2023-03-08 11:55:34,864 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276534864"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276534864"}]},"ts":"1678276534864"} 2023-03-08 11:55:34,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=90 2023-03-08 11:55:34,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=90, state=SUCCESS; CloseRegionProcedure 1b2090f00e0a2d838e28358e6d6297e4, server=jenkins-hbase4.apache.org,45551,1678276414946 in 165 msec 2023-03-08 11:55:34,867 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, UNASSIGN in 170 msec 2023-03-08 11:55:34,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 11:55:34,868 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:34,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:34,869 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:34,870 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=8a2814671292e9b2760956bf02135606, regionState=CLOSED 2023-03-08 11:55:34,870 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276534870"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276534870"}]},"ts":"1678276534870"} 2023-03-08 11:55:34,872 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=91 2023-03-08 11:55:34,872 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=91, state=SUCCESS; CloseRegionProcedure 8a2814671292e9b2760956bf02135606, server=jenkins-hbase4.apache.org,45551,1678276414946 in 172 msec 2023-03-08 11:55:34,873 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=88 2023-03-08 11:55:34,873 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, UNASSIGN in 176 msec 2023-03-08 11:55:34,873 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276534873"}]},"ts":"1678276534873"} 2023-03-08 11:55:34,874 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-08 11:55:34,875 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-08 11:55:34,877 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 187 msec 2023-03-08 11:55:34,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 11:55:34,995 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-08 11:55:35,007 INFO [Listener at localhost/36211] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1678276521850-1678276535007 2023-03-08 11:55:35,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 11:55:35,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678276535008 (current time:1678276535008). 2023-03-08 11:55:35,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-08 11:55:35,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] zookeeper.ReadOnlyZKClient(139): Connect 0x3b9d92ab to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:55:35,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@13d92ead, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:55:35,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:55:35,016 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47236, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:55:35,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3b9d92ab to 127.0.0.1:51863 2023-03-08 11:55:35,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:55:35,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-08 11:55:35,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 11:55:35,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 11:55:35,019 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 11:55:35,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 11:55:35,020 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1678276521850-1678276535007 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-08 11:55:35,020 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678277135020 2023-03-08 11:55:35,020 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:35,021 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-08 11:55:35,026 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-08 11:55:35,028 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 11:55:35,028 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 11:55:35,028 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 11:55:35,028 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 11:55:35,028 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 11:55:35,028 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 11:55:35,028 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 11:55:35,028 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/34dc99c54d59489f8d38971f6dcdc3cc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/f9a342fca44c47d8a8766eecb8e40e68_SeqId_29_] hfiles 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/630e1c12044e45edb608dfe99d90d76a_SeqId_29_, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/c65ac81cb05c4576917d2555d97a3d4a] hfiles 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/34dc99c54d59489f8d38971f6dcdc3cc 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/630e1c12044e45edb608dfe99d90d76a_SeqId_29_ 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/f9a342fca44c47d8a8766eecb8e40e68_SeqId_29_ 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/c65ac81cb05c4576917d2555d97a3d4a 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/747754bd5443434fa3da4893b56ea646, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/9393d516f2c941999c98dd13e993e44b_SeqId_29_] hfiles 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/747754bd5443434fa3da4893b56ea646 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030834a03b91133e4394b759eec8934b6b90, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308cd60b94817224f84b309293e4ed30176, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303082f6ec3af72ad4470917b6c949c92ea92] hfiles 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/9393d516f2c941999c98dd13e993e44b_SeqId_29_ 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030834a03b91133e4394b759eec8934b6b90 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308cd60b94817224f84b309293e4ed30176 2023-03-08 11:55:35,031 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303082f6ec3af72ad4470917b6c949c92ea92 2023-03-08 11:55:35,032 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/fc79ed68bbcc4211a348d8730b95b905] hfiles 2023-03-08 11:55:35,032 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/b9d569129c7c4e7496505f062634856a] hfiles 2023-03-08 11:55:35,032 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/0faf115824b3420db547c12da218e57f] hfiles 2023-03-08 11:55:35,032 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/fc79ed68bbcc4211a348d8730b95b905 2023-03-08 11:55:35,032 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/0faf115824b3420db547c12da218e57f 2023-03-08 11:55:35,032 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/b9d569129c7c4e7496505f062634856a 2023-03-08 11:55:35,037 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308054cd3fdc5b14d87a0cea966f45e9c01, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308576a00e3c6064cb79e69ac1ced33974a, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030870c894d1fe9c45bdb1e8bec2c7c3a173, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308cef6c2bbc100468792820f193ef34319, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308dd803c8ec5b24151bcdb8b045013a73b_del, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030804b635893f374fd9a728fd22f708ce05, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303080f559134c40f47d2b5ae8eb46fbcf448, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030826f1ddad5e534a108c3528f6b6d86c72, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085a7dc8698c054ac382e595c8fd20426b_del, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030861f52ec109df4a86b8a622ce66a69748, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030807304ad827064736bd3bbd01e57050d8, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030808c1a747406644088a4f7542cc0c2930, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030847ce68641dda433f827e422e602aef3d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308a5b12cdd84a64eb280792a364472ded7_del, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d2c5e8e75a784ce1bf17b8c0964b5541] hfiles 2023-03-08 11:55:35,037 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308054cd3fdc5b14d87a0cea966f45e9c01 2023-03-08 11:55:35,037 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308576a00e3c6064cb79e69ac1ced33974a 2023-03-08 11:55:35,037 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030870c894d1fe9c45bdb1e8bec2c7c3a173 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308cef6c2bbc100468792820f193ef34319 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308dd803c8ec5b24151bcdb8b045013a73b_del 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030804b635893f374fd9a728fd22f708ce05 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303080f559134c40f47d2b5ae8eb46fbcf448 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030826f1ddad5e534a108c3528f6b6d86c72 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085a7dc8698c054ac382e595c8fd20426b_del 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023030861f52ec109df4a86b8a622ce66a69748 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030807304ad827064736bd3bbd01e57050d8 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030808c1a747406644088a4f7542cc0c2930 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030847ce68641dda433f827e422e602aef3d 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308a5b12cdd84a64eb280792a364472ded7_del 2023-03-08 11:55:35,038 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308d2c5e8e75a784ce1bf17b8c0964b5541 2023-03-08 11:55:35,121 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:35,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:35,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 11:55:35,122 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-08 11:55:35,322 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:35,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:35,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 11:55:35,323 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-08 11:55:35,444 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-08 11:55:35,444 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1678276521850-1678276535007 2023-03-08 11:55:35,445 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678276521850-1678276535007 2023-03-08 11:55:35,623 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:35,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:35,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 11:55:35,624 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-08 11:55:35,868 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678276521850-1678276535007 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/hbase-failsafe-snaptb-1678276521850-1678276535007 2023-03-08 11:55:36,125 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:36,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:36,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 11:55:36,126 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-08 11:55:36,255 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 11:55:36,693 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1678276521850-1678276535007 of table testCompactionWithHFileLink completed 2023-03-08 11:55:36,693 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678276521850-1678276535007 2023-03-08 11:55:36,693 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678276521850-1678276535007 2023-03-08 11:55:36,693 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1678276521850-1678276535007 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678276535020Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1678276535027 (+7 ms)Consolidate snapshot: hbase-failsafe-snaptb-1678276521850-1678276535007 at 1678276535444 (+417 ms)Loading Region manifests for hbase-failsafe-snaptb-1678276521850-1678276535007 at 1678276535444Writing data manifest for hbase-failsafe-snaptb-1678276521850-1678276535007 at 1678276535448 (+4 ms)Verifying snapshot: hbase-failsafe-snaptb-1678276521850-1678276535007 at 1678276535854 (+406 ms)Snapshot hbase-failsafe-snaptb-1678276521850-1678276535007 of table testCompactionWithHFileLink completed at 1678276536693 (+839 ms) 2023-03-08 11:55:36,694 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 11:55:36,696 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.6760 sec 2023-03-08 11:55:37,126 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 11:55:37,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 11:55:37,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1678276521850-1678276535007 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-08 11:55:37,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678276521850" table: "testCompactionWithHFileLink" creation_time: 1678276521860 type: FLUSH version: 2 ttl: 0 ) 2023-03-08 11:55:37,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1678276521850 as table=testCompactionWithHFileLink 2023-03-08 11:55:37,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 11:55:37,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 11:55:37,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 11:55:37,587 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-08 11:55:37,588 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-08 11:55:37,588 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-08 11:55:37,589 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 11:55:37,593 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1678276521850" table: "testCompactionWithHFileLink" creation_time: 1678276521860 type: FLUSH version: 2 ttl: 0 2023-03-08 11:55:37,594 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink 2023-03-08 11:55:37,598 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-08 11:55:37,601 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308cd60b94817224f84b309293e4ed30176 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308cd60b94817224f84b309293e4ed30176 2023-03-08 11:55:37,603 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303082f6ec3af72ad4470917b6c949c92ea92 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303082f6ec3af72ad4470917b6c949c92ea92 2023-03-08 11:55:37,604 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030834a03b91133e4394b759eec8934b6b90 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030834a03b91133e4394b759eec8934b6b90 2023-03-08 11:55:37,604 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303085a93e00ffcfc4680bd8693f36acefb79 of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,607 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303086765cd9fd5364581b2b5cc820219cb0b of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,610 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023030877dd088b99974eeb8271403356acfffc of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,613 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230308848093d93c8a439596e91454ff9495f2 of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,615 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023030871f36923d1904dce9dc63544f1db105f of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,618 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303088f1203eb82d14d86b18c22fd2a4819de of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,621 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230308abdbea5c3a9049b4a8f1ee86acbfe7f0 of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,623 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230308c7c09d3168c34702afb2096403f4dbad of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,626 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303082aec09723e744fe089858f4c25d2aa10 of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,629 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303085e92c651e62347979ce171cd2b619c48 of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,633 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023030893ca3401786d4c4bb7ef4ff0c678f2d2 of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,636 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230308cafe3e79be0c49159be042b6efb707f0 of snapshot=snaptb-1678276521850 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,642 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308dd803c8ec5b24151bcdb8b045013a73b_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308dd803c8ec5b24151bcdb8b045013a73b_del 2023-03-08 11:55:37,643 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308a5b12cdd84a64eb280792a364472ded7_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308a5b12cdd84a64eb280792a364472ded7_del 2023-03-08 11:55:37,644 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085a7dc8698c054ac382e595c8fd20426b_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085a7dc8698c054ac382e595c8fd20426b_del 2023-03-08 11:55:37,644 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:37,644 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:37,644 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:37,648 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/34dc99c54d59489f8d38971f6dcdc3cc to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/34dc99c54d59489f8d38971f6dcdc3cc 2023-03-08 11:55:37,648 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/747754bd5443434fa3da4893b56ea646 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/747754bd5443434fa3da4893b56ea646 2023-03-08 11:55:37,648 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/630e1c12044e45edb608dfe99d90d76a_SeqId_29_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/630e1c12044e45edb608dfe99d90d76a_SeqId_29_ 2023-03-08 11:55:37,649 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/9393d516f2c941999c98dd13e993e44b_SeqId_29_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/9393d516f2c941999c98dd13e993e44b_SeqId_29_ 2023-03-08 11:55:37,649 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3356b1ae0f4c4f8996df9885e552ca20 of snapshot=snaptb-1678276521850 to region=1b2090f00e0a2d838e28358e6d6297e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,649 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/c65ac81cb05c4576917d2555d97a3d4a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/c65ac81cb05c4576917d2555d97a3d4a 2023-03-08 11:55:37,649 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 95af003fa2434171b6ecd72bae8a1c8e of snapshot=snaptb-1678276521850 to region=bb0058b62005e791ee17613797cadc96 table=testCompactionWithHFileLink 2023-03-08 11:55:37,649 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/f9a342fca44c47d8a8766eecb8e40e68_SeqId_29_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/f9a342fca44c47d8a8766eecb8e40e68_SeqId_29_ 2023-03-08 11:55:37,649 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 12fb73322f2b462daa9f7f2b5e2177a2 of snapshot=snaptb-1678276521850 to region=8a2814671292e9b2760956bf02135606 table=testCompactionWithHFileLink 2023-03-08 11:55:37,653 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 26c74f6094ab4fb1a736236655bf5017 of snapshot=snaptb-1678276521850 to region=bb0058b62005e791ee17613797cadc96 table=testCompactionWithHFileLink 2023-03-08 11:55:37,653 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dbcdd7d7d25445c094214456a69dcade of snapshot=snaptb-1678276521850 to region=1b2090f00e0a2d838e28358e6d6297e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,653 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 051d4a6cbfe34e7a96dc643c2b20cf3a of snapshot=snaptb-1678276521850 to region=8a2814671292e9b2760956bf02135606 table=testCompactionWithHFileLink 2023-03-08 11:55:37,660 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink bc161e308bab4d49a0b696e98fc1727c of snapshot=snaptb-1678276521850 to region=1b2090f00e0a2d838e28358e6d6297e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,661 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 27d2cdabb3144f8dad476aebd9d2ba92 of snapshot=snaptb-1678276521850 to region=bb0058b62005e791ee17613797cadc96 table=testCompactionWithHFileLink 2023-03-08 11:55:37,661 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1cd5f0aa679d4c86b010d1eb4ea4235c of snapshot=snaptb-1678276521850 to region=8a2814671292e9b2760956bf02135606 table=testCompactionWithHFileLink 2023-03-08 11:55:37,672 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3194c7487b0945148bb92e7a12c01a06 of snapshot=snaptb-1678276521850 to region=bb0058b62005e791ee17613797cadc96 table=testCompactionWithHFileLink 2023-03-08 11:55:37,676 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4183389884c8402d912826df5bd5dbfe of snapshot=snaptb-1678276521850 to region=1b2090f00e0a2d838e28358e6d6297e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,676 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e99f9377a8d64fbf9813721496bc314c of snapshot=snaptb-1678276521850 to region=8a2814671292e9b2760956bf02135606 table=testCompactionWithHFileLink 2023-03-08 11:55:37,680 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/fc79ed68bbcc4211a348d8730b95b905 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/fc79ed68bbcc4211a348d8730b95b905 2023-03-08 11:55:37,680 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b07367b5aace48be8cff48334d01f5fc of snapshot=snaptb-1678276521850 to region=bb0058b62005e791ee17613797cadc96 table=testCompactionWithHFileLink 2023-03-08 11:55:37,682 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/0faf115824b3420db547c12da218e57f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/0faf115824b3420db547c12da218e57f 2023-03-08 11:55:37,682 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9d982a00f7234f3799eff4331f7781ea of snapshot=snaptb-1678276521850 to region=1b2090f00e0a2d838e28358e6d6297e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,682 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/b9d569129c7c4e7496505f062634856a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/b9d569129c7c4e7496505f062634856a 2023-03-08 11:55:37,682 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9c92d16d2b9f43529ed6b665e31e6836 of snapshot=snaptb-1678276521850 to region=8a2814671292e9b2760956bf02135606 table=testCompactionWithHFileLink 2023-03-08 11:55:37,684 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 81110e19fe8549e99c21702c394f377b of snapshot=snaptb-1678276521850 to region=bb0058b62005e791ee17613797cadc96 table=testCompactionWithHFileLink 2023-03-08 11:55:37,685 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b4ac784cd164469a8b3459a552bab0db of snapshot=snaptb-1678276521850 to region=1b2090f00e0a2d838e28358e6d6297e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,686 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3a0c4db82b484bbea43a20cc9970f523 of snapshot=snaptb-1678276521850 to region=8a2814671292e9b2760956bf02135606 table=testCompactionWithHFileLink 2023-03-08 11:55:37,687 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 450acdbf2fdd4380b22b172baf63edb9 of snapshot=snaptb-1678276521850 to region=bb0058b62005e791ee17613797cadc96 table=testCompactionWithHFileLink 2023-03-08 11:55:37,689 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3e8179efb3d04e28ab09e2d11966c946 of snapshot=snaptb-1678276521850 to region=1b2090f00e0a2d838e28358e6d6297e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,689 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 855e4339fb014cda9a4a8621eb2c9cc5 of snapshot=snaptb-1678276521850 to region=8a2814671292e9b2760956bf02135606 table=testCompactionWithHFileLink 2023-03-08 11:55:37,690 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1f73d40af99a49ff99f80f52c56f4085 of snapshot=snaptb-1678276521850 to region=bb0058b62005e791ee17613797cadc96 table=testCompactionWithHFileLink 2023-03-08 11:55:37,692 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5908c3bffaf24fcaa736f19e125a250b of snapshot=snaptb-1678276521850 to region=1b2090f00e0a2d838e28358e6d6297e4 table=testCompactionWithHFileLink 2023-03-08 11:55:37,692 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5a3c6c586c1d4a2da00c3fe70280debf of snapshot=snaptb-1678276521850 to region=8a2814671292e9b2760956bf02135606 table=testCompactionWithHFileLink 2023-03-08 11:55:37,695 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1678276521850" table: "testCompactionWithHFileLink" creation_time: 1678276521860 type: FLUSH version: 2 ttl: 0 2023-03-08 11:55:37,697 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678276537696"}]},"ts":"9223372036854775807"} 2023-03-08 11:55:37,697 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678276537696"}]},"ts":"9223372036854775807"} 2023-03-08 11:55:37,697 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678276537696"}]},"ts":"9223372036854775807"} 2023-03-08 11:55:37,698 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-08 11:55:37,699 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1b2090f00e0a2d838e28358e6d6297e4, NAME => 'testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => 8a2814671292e9b2760956bf02135606, NAME => 'testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => bb0058b62005e791ee17613797cadc96, NAME => 'testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.', STARTKEY => '', ENDKEY => '1'}] 2023-03-08 11:55:37,699 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276537697"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276537697"}]},"ts":"1678276537697"} 2023-03-08 11:55:37,699 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276537697"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276537697"}]},"ts":"1678276537697"} 2023-03-08 11:55:37,699 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276537697"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276537697"}]},"ts":"1678276537697"} 2023-03-08 11:55:37,700 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 11:55:37,700 INFO [PEWorker-3] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-08 11:55:37,700 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 1b2090f00e0a2d838e28358e6d6297e4, NAME => 'testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => 8a2814671292e9b2760956bf02135606, NAME => 'testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => bb0058b62005e791ee17613797cadc96, NAME => 'testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.', STARTKEY => '', ENDKEY => '1'}] 2023-03-08 11:55:37,701 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1678276521850 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-08 11:55:37,702 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678276521850" table: "testCompactionWithHFileLink" creation_time: 1678276521860 type: FLUSH version: 2 ttl: 0 ) in 538 msec 2023-03-08 11:55:37,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 11:55:37,784 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-08 11:55:37,784 INFO [Listener at localhost/36211] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1678276521850-1678276535007 2023-03-08 11:55:37,788 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1678276521850-1678276535007" 2023-03-08 11:55:37,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1678276521850-1678276535007 2023-03-08 11:55:37,793 INFO [Listener at localhost/36211] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-08 11:55:37,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-03-08 11:55:37,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-08 11:55:37,805 INFO [PEWorker-2] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-08 11:55:37,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 11:55:37,805 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276537805"}]},"ts":"1678276537805"} 2023-03-08 11:55:37,806 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-08 11:55:37,808 DEBUG [PEWorker-2] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-08 11:55:37,808 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, ASSIGN}] 2023-03-08 11:55:37,809 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, ASSIGN 2023-03-08 11:55:37,809 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, ASSIGN 2023-03-08 11:55:37,810 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, ASSIGN 2023-03-08 11:55:37,810 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 11:55:37,810 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 11:55:37,810 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 11:55:37,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 11:55:37,962 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=8a2814671292e9b2760956bf02135606, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:37,962 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=1b2090f00e0a2d838e28358e6d6297e4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:37,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=bb0058b62005e791ee17613797cadc96, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:37,962 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276537962"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276537962"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276537962"}]},"ts":"1678276537962"} 2023-03-08 11:55:37,962 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276537962"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276537962"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276537962"}]},"ts":"1678276537962"} 2023-03-08 11:55:37,962 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276537962"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276537962"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276537962"}]},"ts":"1678276537962"} 2023-03-08 11:55:37,963 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure 8a2814671292e9b2760956bf02135606, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:37,964 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=99, state=RUNNABLE; OpenRegionProcedure 1b2090f00e0a2d838e28358e6d6297e4, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:37,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure bb0058b62005e791ee17613797cadc96, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:38,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 11:55:38,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:38,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bb0058b62005e791ee17613797cadc96, NAME => 'testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 11:55:38,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:38,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:38,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:38,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:38,119 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:38,120 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1 2023-03-08 11:55:38,120 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1 2023-03-08 11:55:38,120 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb0058b62005e791ee17613797cadc96 columnFamilyName family1 2023-03-08 11:55:38,131 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-26c74f6094ab4fb1a736236655bf5017 2023-03-08 11:55:38,136 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-27d2cdabb3144f8dad476aebd9d2ba92 2023-03-08 11:55:38,141 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-3194c7487b0945148bb92e7a12c01a06 2023-03-08 11:55:38,145 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-95af003fa2434171b6ecd72bae8a1c8e 2023-03-08 11:55:38,145 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(310): Store=bb0058b62005e791ee17613797cadc96/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:38,146 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:38,146 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2 2023-03-08 11:55:38,146 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2 2023-03-08 11:55:38,147 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb0058b62005e791ee17613797cadc96 columnFamilyName family2 2023-03-08 11:55:38,153 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-1f73d40af99a49ff99f80f52c56f4085 2023-03-08 11:55:38,157 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-450acdbf2fdd4380b22b172baf63edb9 2023-03-08 11:55:38,162 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-81110e19fe8549e99c21702c394f377b 2023-03-08 11:55:38,167 DEBUG [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-b07367b5aace48be8cff48334d01f5fc 2023-03-08 11:55:38,167 INFO [StoreOpener-bb0058b62005e791ee17613797cadc96-1] regionserver.HStore(310): Store=bb0058b62005e791ee17613797cadc96/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:38,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:38,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:38,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:55:38,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bb0058b62005e791ee17613797cadc96 2023-03-08 11:55:38,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bb0058b62005e791ee17613797cadc96; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66164440, jitterRate=-0.014073014259338379}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:55:38,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:38,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96., pid=103, masterSystemTime=1678276538115 2023-03-08 11:55:38,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:55:38,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-08 11:55:38,174 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 11:55:38,174 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 11:55:38,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:38,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:38,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:38,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8a2814671292e9b2760956bf02135606, NAME => 'testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 11:55:38,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:38,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:38,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:38,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:38,177 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=bb0058b62005e791ee17613797cadc96, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:38,177 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276538177"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276538177"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276538177"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276538177"}]},"ts":"1678276538177"} 2023-03-08 11:55:38,177 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 11:55:38,177 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HStore(1912): bb0058b62005e791ee17613797cadc96/family2 is initiating minor compaction (all files) 2023-03-08 11:55:38,178 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HRegion(2259): Starting compaction of bb0058b62005e791ee17613797cadc96/family2 in testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:38,178 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 11:55:38,178 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-b07367b5aace48be8cff48334d01f5fc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-81110e19fe8549e99c21702c394f377b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-450acdbf2fdd4380b22b172baf63edb9, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-1f73d40af99a49ff99f80f52c56f4085] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp, totalSize=20.4 K 2023-03-08 11:55:38,178 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1912): bb0058b62005e791ee17613797cadc96/family1 is initiating minor compaction (all files) 2023-03-08 11:55:38,178 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:38,178 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of bb0058b62005e791ee17613797cadc96/family1 in testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:55:38,178 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-95af003fa2434171b6ecd72bae8a1c8e, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-26c74f6094ab4fb1a736236655bf5017, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-27d2cdabb3144f8dad476aebd9d2ba92, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-3194c7487b0945148bb92e7a12c01a06] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp, totalSize=21.6 K 2023-03-08 11:55:38,178 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-b07367b5aace48be8cff48334d01f5fc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514609 2023-03-08 11:55:38,179 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1 2023-03-08 11:55:38,179 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1 2023-03-08 11:55:38,179 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-95af003fa2434171b6ecd72bae8a1c8e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514609 2023-03-08 11:55:38,179 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a2814671292e9b2760956bf02135606 columnFamilyName family1 2023-03-08 11:55:38,179 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-81110e19fe8549e99c21702c394f377b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516444 2023-03-08 11:55:38,180 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=98 2023-03-08 11:55:38,180 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-26c74f6094ab4fb1a736236655bf5017, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516444 2023-03-08 11:55:38,181 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure bb0058b62005e791ee17613797cadc96, server=jenkins-hbase4.apache.org,45551,1678276414946 in 214 msec 2023-03-08 11:55:38,181 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-450acdbf2fdd4380b22b172baf63edb9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518238 2023-03-08 11:55:38,181 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-27d2cdabb3144f8dad476aebd9d2ba92, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518238 2023-03-08 11:55:38,181 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-1f73d40af99a49ff99f80f52c56f4085, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:38,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb0058b62005e791ee17613797cadc96, ASSIGN in 373 msec 2023-03-08 11:55:38,182 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-3194c7487b0945148bb92e7a12c01a06, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:38,192 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-051d4a6cbfe34e7a96dc643c2b20cf3a 2023-03-08 11:55:38,209 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-12fb73322f2b462daa9f7f2b5e2177a2 2023-03-08 11:55:38,236 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): bb0058b62005e791ee17613797cadc96#family1#compaction#113 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:38,236 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-1cd5f0aa679d4c86b010d1eb4ea4235c 2023-03-08 11:55:38,240 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] throttle.PressureAwareThroughputController(145): bb0058b62005e791ee17613797cadc96#family2#compaction#112 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:38,258 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-e99f9377a8d64fbf9813721496bc314c 2023-03-08 11:55:38,258 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(310): Store=8a2814671292e9b2760956bf02135606/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:38,258 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:38,259 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2 2023-03-08 11:55:38,259 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2 2023-03-08 11:55:38,259 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a2814671292e9b2760956bf02135606 columnFamilyName family2 2023-03-08 11:55:38,268 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-3a0c4db82b484bbea43a20cc9970f523 2023-03-08 11:55:38,272 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-5a3c6c586c1d4a2da00c3fe70280debf 2023-03-08 11:55:38,278 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-855e4339fb014cda9a4a8621eb2c9cc5 2023-03-08 11:55:38,282 DEBUG [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-9c92d16d2b9f43529ed6b665e31e6836 2023-03-08 11:55:38,282 INFO [StoreOpener-8a2814671292e9b2760956bf02135606-1] regionserver.HStore(310): Store=8a2814671292e9b2760956bf02135606/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:38,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606 2023-03-08 11:55:38,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606 2023-03-08 11:55:38,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:55:38,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8a2814671292e9b2760956bf02135606 2023-03-08 11:55:38,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8a2814671292e9b2760956bf02135606; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63356572, jitterRate=-0.05591350793838501}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:55:38,287 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:38,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606., pid=101, masterSystemTime=1678276538115 2023-03-08 11:55:38,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-08 11:55:38,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-08 11:55:38,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:38,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:38,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:38,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b2090f00e0a2d838e28358e6d6297e4, NAME => 'testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 11:55:38,290 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=8a2814671292e9b2760956bf02135606, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:38,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:38,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:38,290 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678276538290"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276538290"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276538290"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276538290"}]},"ts":"1678276538290"} 2023-03-08 11:55:38,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:38,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:38,293 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:38,294 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1 2023-03-08 11:55:38,294 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1 2023-03-08 11:55:38,294 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b2090f00e0a2d838e28358e6d6297e4 columnFamilyName family1 2023-03-08 11:55:38,295 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-03-08 11:55:38,295 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure 8a2814671292e9b2760956bf02135606, server=jenkins-hbase4.apache.org,45551,1678276414946 in 330 msec 2023-03-08 11:55:38,296 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=8a2814671292e9b2760956bf02135606, ASSIGN in 487 msec 2023-03-08 11:55:38,301 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3356b1ae0f4c4f8996df9885e552ca20 2023-03-08 11:55:38,309 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-4183389884c8402d912826df5bd5dbfe 2023-03-08 11:55:38,314 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-bc161e308bab4d49a0b696e98fc1727c 2023-03-08 11:55:38,322 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-dbcdd7d7d25445c094214456a69dcade 2023-03-08 11:55:38,322 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(310): Store=1b2090f00e0a2d838e28358e6d6297e4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:38,323 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:38,324 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2 2023-03-08 11:55:38,324 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2 2023-03-08 11:55:38,324 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b2090f00e0a2d838e28358e6d6297e4 columnFamilyName family2 2023-03-08 11:55:38,340 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3e8179efb3d04e28ab09e2d11966c946 2023-03-08 11:55:38,348 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-5908c3bffaf24fcaa736f19e125a250b 2023-03-08 11:55:38,354 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-9d982a00f7234f3799eff4331f7781ea 2023-03-08 11:55:38,360 DEBUG [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(539): loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-b4ac784cd164469a8b3459a552bab0db 2023-03-08 11:55:38,361 INFO [StoreOpener-1b2090f00e0a2d838e28358e6d6297e4-1] regionserver.HStore(310): Store=1b2090f00e0a2d838e28358e6d6297e4/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:38,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:38,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:38,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:55:38,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:55:38,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b2090f00e0a2d838e28358e6d6297e4; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75416990, jitterRate=0.12380072474479675}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:55:38,366 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:38,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4., pid=102, masterSystemTime=1678276538115 2023-03-08 11:55:38,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-08 11:55:38,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-08 11:55:38,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:38,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:38,369 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=1b2090f00e0a2d838e28358e6d6297e4, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:38,370 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276538369"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276538369"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276538369"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276538369"}]},"ts":"1678276538369"} 2023-03-08 11:55:38,372 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=99 2023-03-08 11:55:38,372 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=99, state=SUCCESS; OpenRegionProcedure 1b2090f00e0a2d838e28358e6d6297e4, server=jenkins-hbase4.apache.org,45551,1678276414946 in 407 msec 2023-03-08 11:55:38,373 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-03-08 11:55:38,373 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1b2090f00e0a2d838e28358e6d6297e4, ASSIGN in 564 msec 2023-03-08 11:55:38,374 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276538374"}]},"ts":"1678276538374"} 2023-03-08 11:55:38,375 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-08 11:55:38,376 INFO [PEWorker-1] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-08 11:55:38,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 579 msec 2023-03-08 11:55:38,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 11:55:38,408 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-08 11:55:38,419 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:38,427 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:38,433 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:38,434 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:38,439 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 11:55:38,452 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-08 11:55:38,452 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:38,452 DEBUG [Listener at localhost/36211] zookeeper.ReadOnlyZKClient(139): Connect 0x3865866b to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:55:38,457 DEBUG [Listener at localhost/36211] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6f90ddc8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:55:38,458 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:38,458 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:38,458 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:38,550 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308252c3c5300a7476caff196a520fa5ddd to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308252c3c5300a7476caff196a520fa5ddd 2023-03-08 11:55:38,551 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:38,570 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:55:38,574 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47242, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:55:38,580 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:55:38,583 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60228, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:55:38,593 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6b476a8bf6a449b798e4f7d06ebdf766 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-08 11:55:38,593 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=32 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6b476a8bf6a449b798e4f7d06ebdf766}] 2023-03-08 11:55:38,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6b476a8bf6a449b798e4f7d06ebdf766 for inclusion in bb0058b62005e791ee17613797cadc96/family1 2023-03-08 11:55:38,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-08 11:55:38,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 11:55:38,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:38,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6b476a8bf6a449b798e4f7d06ebdf766 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__ge15h6n5bh2j2gl7sakd7ckros6l9gsvn7aru4ceb0t3u92li1k2dkrij1emogi/family1/6b476a8bf6a449b798e4f7d06ebdf766 2023-03-08 11:55:38,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__ge15h6n5bh2j2gl7sakd7ckros6l9gsvn7aru4ceb0t3u92li1k2dkrij1emogi/family1/6b476a8bf6a449b798e4f7d06ebdf766 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/9826a9bfd22945cba816a4a130750b94_SeqId_34_ 2023-03-08 11:55:38,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__ge15h6n5bh2j2gl7sakd7ckros6l9gsvn7aru4ceb0t3u92li1k2dkrij1emogi/family1/6b476a8bf6a449b798e4f7d06ebdf766 into bb0058b62005e791ee17613797cadc96/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/9826a9bfd22945cba816a4a130750b94_SeqId_34_ - updating store file list. 2023-03-08 11:55:38,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/9826a9bfd22945cba816a4a130750b94_SeqId_34_ into bb0058b62005e791ee17613797cadc96/family1 2023-03-08 11:55:38,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__ge15h6n5bh2j2gl7sakd7ckros6l9gsvn7aru4ceb0t3u92li1k2dkrij1emogi/family1/6b476a8bf6a449b798e4f7d06ebdf766 into bb0058b62005e791ee17613797cadc96/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/9826a9bfd22945cba816a4a130750b94_SeqId_34_) 2023-03-08 11:55:38,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__ge15h6n5bh2j2gl7sakd7ckros6l9gsvn7aru4ceb0t3u92li1k2dkrij1emogi/family1/6b476a8bf6a449b798e4f7d06ebdf766 2023-03-08 11:55:38,609 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:38,610 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:38,611 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303082aec09723e744fe089858f4c25d2aa10 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303082aec09723e744fe089858f4c25d2aa10 2023-03-08 11:55:38,612 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303085e92c651e62347979ce171cd2b619c48 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303085e92c651e62347979ce171cd2b619c48 2023-03-08 11:55:38,613 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023030893ca3401786d4c4bb7ef4ff0c678f2d2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023030893ca3401786d4c4bb7ef4ff0c678f2d2 2023-03-08 11:55:38,613 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308cafe3e79be0c49159be042b6efb707f0 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308cafe3e79be0c49159be042b6efb707f0 2023-03-08 11:55:38,614 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 11:55:38,614 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 11:55:38,614 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:39,352 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308e6c0f8841b3348fb8093e696e576ea6d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308e6c0f8841b3348fb8093e696e576ea6d 2023-03-08 11:55:39,352 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303081069d104fd3d42c99855a9761d10b76d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303081069d104fd3d42c99855a9761d10b76d 2023-03-08 11:55:39,353 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:39,354 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:39,377 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/dcb29d0686fa48c59cc54ff2879867f9 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-08 11:55:39,378 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/dcb29d0686fa48c59cc54ff2879867f9}] 2023-03-08 11:55:39,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/dcb29d0686fa48c59cc54ff2879867f9 for inclusion in 1b2090f00e0a2d838e28358e6d6297e4/family1 2023-03-08 11:55:39,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-08 11:55:39,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 11:55:39,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:39,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/dcb29d0686fa48c59cc54ff2879867f9 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__b6m4tn1ved65lf6c51r2lkl8r1644p38nq3jfkrmt7o16gfs3jqgbq02o5u5nks3/family1/dcb29d0686fa48c59cc54ff2879867f9 2023-03-08 11:55:39,386 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/78ce381e728d4bddb76905cd2e2d485e first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-08 11:55:39,387 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/78ce381e728d4bddb76905cd2e2d485e}] 2023-03-08 11:55:39,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__b6m4tn1ved65lf6c51r2lkl8r1644p38nq3jfkrmt7o16gfs3jqgbq02o5u5nks3/family1/dcb29d0686fa48c59cc54ff2879867f9 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/a478acbc799944c8afb63b82457fe276_SeqId_34_ 2023-03-08 11:55:39,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__b6m4tn1ved65lf6c51r2lkl8r1644p38nq3jfkrmt7o16gfs3jqgbq02o5u5nks3/family1/dcb29d0686fa48c59cc54ff2879867f9 into 1b2090f00e0a2d838e28358e6d6297e4/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/a478acbc799944c8afb63b82457fe276_SeqId_34_ - updating store file list. 2023-03-08 11:55:39,389 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/78ce381e728d4bddb76905cd2e2d485e for inclusion in 8a2814671292e9b2760956bf02135606/family1 2023-03-08 11:55:39,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-08 11:55:39,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 11:55:39,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:39,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/a478acbc799944c8afb63b82457fe276_SeqId_34_ into 1b2090f00e0a2d838e28358e6d6297e4/family1 2023-03-08 11:55:39,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__b6m4tn1ved65lf6c51r2lkl8r1644p38nq3jfkrmt7o16gfs3jqgbq02o5u5nks3/family1/dcb29d0686fa48c59cc54ff2879867f9 into 1b2090f00e0a2d838e28358e6d6297e4/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/a478acbc799944c8afb63b82457fe276_SeqId_34_) 2023-03-08 11:55:39,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__b6m4tn1ved65lf6c51r2lkl8r1644p38nq3jfkrmt7o16gfs3jqgbq02o5u5nks3/family1/dcb29d0686fa48c59cc54ff2879867f9 2023-03-08 11:55:39,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:39,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/78ce381e728d4bddb76905cd2e2d485e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__pqcke2kb44281uagk520t47c0431768bk309gg0epvq3t10r3k5877dlea47tbcc/family1/78ce381e728d4bddb76905cd2e2d485e 2023-03-08 11:55:39,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__pqcke2kb44281uagk520t47c0431768bk309gg0epvq3t10r3k5877dlea47tbcc/family1/78ce381e728d4bddb76905cd2e2d485e as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/eaf94b0f8129492cae1b45247aa8d7a5_SeqId_34_ 2023-03-08 11:55:39,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-08 11:55:39,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): 1b2090f00e0a2d838e28358e6d6297e4/family1 is initiating minor compaction (all files) 2023-03-08 11:55:39,395 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__pqcke2kb44281uagk520t47c0431768bk309gg0epvq3t10r3k5877dlea47tbcc/family1/78ce381e728d4bddb76905cd2e2d485e into 8a2814671292e9b2760956bf02135606/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/eaf94b0f8129492cae1b45247aa8d7a5_SeqId_34_ - updating store file list. 2023-03-08 11:55:39,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@490ce6e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-08 11:55:39,397 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:39,397 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:39,399 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/eaf94b0f8129492cae1b45247aa8d7a5_SeqId_34_ into 8a2814671292e9b2760956bf02135606/family1 2023-03-08 11:55:39,399 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__pqcke2kb44281uagk520t47c0431768bk309gg0epvq3t10r3k5877dlea47tbcc/family1/78ce381e728d4bddb76905cd2e2d485e into 8a2814671292e9b2760956bf02135606/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/eaf94b0f8129492cae1b45247aa8d7a5_SeqId_34_) 2023-03-08 11:55:39,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testCompactionWithHFileLink__pqcke2kb44281uagk520t47c0431768bk309gg0epvq3t10r3k5877dlea47tbcc/family1/78ce381e728d4bddb76905cd2e2d485e 2023-03-08 11:55:39,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:39,399 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303085a93e00ffcfc4680bd8693f36acefb79 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303085a93e00ffcfc4680bd8693f36acefb79 2023-03-08 11:55:39,401 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303086765cd9fd5364581b2b5cc820219cb0b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303086765cd9fd5364581b2b5cc820219cb0b 2023-03-08 11:55:39,402 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023030877dd088b99974eeb8271403356acfffc to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023030877dd088b99974eeb8271403356acfffc 2023-03-08 11:55:39,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-08 11:55:39,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(1912): 8a2814671292e9b2760956bf02135606/family1 is initiating minor compaction (all files) 2023-03-08 11:55:39,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5d537205; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-03-08 11:55:39,403 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308848093d93c8a439596e91454ff9495f2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308848093d93c8a439596e91454ff9495f2 2023-03-08 11:55:39,403 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 11:55:39,403 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 11:55:39,403 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:39,404 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:39,405 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:39,406 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023030871f36923d1904dce9dc63544f1db105f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023030871f36923d1904dce9dc63544f1db105f 2023-03-08 11:55:39,408 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303088f1203eb82d14d86b18c22fd2a4819de to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303088f1203eb82d14d86b18c22fd2a4819de 2023-03-08 11:55:39,409 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308abdbea5c3a9049b4a8f1ee86acbfe7f0 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308abdbea5c3a9049b4a8f1ee86acbfe7f0 2023-03-08 11:55:39,410 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308c7c09d3168c34702afb2096403f4dbad to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308c7c09d3168c34702afb2096403f4dbad 2023-03-08 11:55:39,410 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 11:55:39,410 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 11:55:39,410 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 11:55:39,410 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 11:55:39,413 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:39,416 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:39,418 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:39,419 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:39,425 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:39,426 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:39,442 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=285 (was 245) - Thread LEAK? -, OpenFileDescriptor=769 (was 624) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=171 (was 159) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=7403 (was 7492) 2023-03-08 11:55:39,454 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=285, OpenFileDescriptor=769, MaxFileDescriptor=60000, SystemLoadAverage=171, ProcessCount=170, AvailableMemoryMB=7402 2023-03-08 11:55:39,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-03-08 11:55:39,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-08 11:55:39,464 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family1/0c7f624d4a1d4ee6b9f4ba3dfd87cd08 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/0c7f624d4a1d4ee6b9f4ba3dfd87cd08 2023-03-08 11:55:39,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-08 11:55:39,467 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 11:55:39,468 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-08 11:55:39,469 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 9 msec 2023-03-08 11:55:39,469 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in bb0058b62005e791ee17613797cadc96/family1 of bb0058b62005e791ee17613797cadc96 into 0c7f624d4a1d4ee6b9f4ba3dfd87cd08(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:39,469 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/.tmp/family2/dca0fdf7c76a461395a78c412768c572 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/dca0fdf7c76a461395a78c412768c572 2023-03-08 11:55:39,469 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:39,469 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96., storeName=bb0058b62005e791ee17613797cadc96/family1, priority=12, startTime=1678276538173; duration=1sec 2023-03-08 11:55:39,470 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-03-08 11:55:39,470 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1b2090f00e0a2d838e28358e6d6297e4/family1 in testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:39,470 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3356b1ae0f4c4f8996df9885e552ca20, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-dbcdd7d7d25445c094214456a69dcade, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-bc161e308bab4d49a0b696e98fc1727c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-4183389884c8402d912826df5bd5dbfe, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/a478acbc799944c8afb63b82457fe276_SeqId_34_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp, totalSize=28.9 K 2023-03-08 11:55:39,471 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3356b1ae0f4c4f8996df9885e552ca20, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514610 2023-03-08 11:55:39,471 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-dbcdd7d7d25445c094214456a69dcade, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516445 2023-03-08 11:55:39,472 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-bc161e308bab4d49a0b696e98fc1727c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518238 2023-03-08 11:55:39,473 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-4183389884c8402d912826df5bd5dbfe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:39,473 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting a478acbc799944c8afb63b82457fe276_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678276514610 2023-03-08 11:55:39,475 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in bb0058b62005e791ee17613797cadc96/family2 of bb0058b62005e791ee17613797cadc96 into dca0fdf7c76a461395a78c412768c572(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:39,475 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:55:39,475 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96., storeName=bb0058b62005e791ee17613797cadc96/family2, priority=12, startTime=1678276538173; duration=1sec 2023-03-08 11:55:39,475 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-08 11:55:39,475 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 8a2814671292e9b2760956bf02135606/family1 in testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:39,475 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-12fb73322f2b462daa9f7f2b5e2177a2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-051d4a6cbfe34e7a96dc643c2b20cf3a, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-1cd5f0aa679d4c86b010d1eb4ea4235c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-e99f9377a8d64fbf9813721496bc314c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/eaf94b0f8129492cae1b45247aa8d7a5_SeqId_34_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp, totalSize=28.9 K 2023-03-08 11:55:39,476 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-12fb73322f2b462daa9f7f2b5e2177a2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514611 2023-03-08 11:55:39,476 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-051d4a6cbfe34e7a96dc643c2b20cf3a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516445 2023-03-08 11:55:39,477 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-1cd5f0aa679d4c86b010d1eb4ea4235c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518239 2023-03-08 11:55:39,478 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-e99f9377a8d64fbf9813721496bc314c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:39,478 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting eaf94b0f8129492cae1b45247aa8d7a5_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678276514611 2023-03-08 11:55:39,481 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-95af003fa2434171b6ecd72bae8a1c8e, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-26c74f6094ab4fb1a736236655bf5017, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-27d2cdabb3144f8dad476aebd9d2ba92, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-3194c7487b0945148bb92e7a12c01a06] to archive 2023-03-08 11:55:39,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:39,483 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-95af003fa2434171b6ecd72bae8a1c8e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-95af003fa2434171b6ecd72bae8a1c8e 2023-03-08 11:55:39,485 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-26c74f6094ab4fb1a736236655bf5017 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-26c74f6094ab4fb1a736236655bf5017 2023-03-08 11:55:39,486 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-27d2cdabb3144f8dad476aebd9d2ba92 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-27d2cdabb3144f8dad476aebd9d2ba92 2023-03-08 11:55:39,488 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-3194c7487b0945148bb92e7a12c01a06 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family1/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-3194c7487b0945148bb92e7a12c01a06 2023-03-08 11:55:39,489 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-b07367b5aace48be8cff48334d01f5fc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-81110e19fe8549e99c21702c394f377b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-450acdbf2fdd4380b22b172baf63edb9, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-1f73d40af99a49ff99f80f52c56f4085] to archive 2023-03-08 11:55:39,490 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:39,492 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-b07367b5aace48be8cff48334d01f5fc to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-b07367b5aace48be8cff48334d01f5fc 2023-03-08 11:55:39,494 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-81110e19fe8549e99c21702c394f377b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-81110e19fe8549e99c21702c394f377b 2023-03-08 11:55:39,495 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-450acdbf2fdd4380b22b172baf63edb9 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-450acdbf2fdd4380b22b172baf63edb9 2023-03-08 11:55:39,496 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1b2090f00e0a2d838e28358e6d6297e4#family1#compaction#114 average throughput is 0.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:39,496 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-1f73d40af99a49ff99f80f52c56f4085 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/family2/testCompactionWithHFileLink=bb0058b62005e791ee17613797cadc96-1f73d40af99a49ff99f80f52c56f4085 2023-03-08 11:55:39,502 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] throttle.PressureAwareThroughputController(145): 8a2814671292e9b2760956bf02135606#family1#compaction#115 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:39,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-08 11:55:39,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 11:55:39,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-08 11:55:39,569 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:55:39,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-08 11:55:39,570 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:55:39,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 11:55:39,571 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:55:39,571 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:55:39,571 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9 2023-03-08 11:55:39,572 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179 empty. 2023-03-08 11:55:39,572 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb empty. 2023-03-08 11:55:39,572 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9 empty. 2023-03-08 11:55:39,572 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:55:39,572 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:55:39,572 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9 2023-03-08 11:55:39,572 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-08 11:55:39,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 11:55:39,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 11:55:39,979 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-08 11:55:39,981 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => b617fde530c0c1bebf3fa2fa6eab77bb, NAME => 'ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:55:39,981 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6249f9a64ffef0dbc4b217b148070179, NAME => 'ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:55:39,981 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 64864abb7d0098b010ede90e39027ce9, NAME => 'ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:55:39,989 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:39,989 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 6249f9a64ffef0dbc4b217b148070179, disabling compactions & flushes 2023-03-08 11:55:39,989 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:39,989 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:39,989 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. after waiting 0 ms 2023-03-08 11:55:39,989 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:39,989 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:39,990 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:55:39,990 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:39,990 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing b617fde530c0c1bebf3fa2fa6eab77bb, disabling compactions & flushes 2023-03-08 11:55:39,990 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:39,990 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:39,990 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. after waiting 0 ms 2023-03-08 11:55:39,990 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:39,990 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:39,990 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:55:40,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 11:55:40,321 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family1/de43a640083c49b19a197347ea969a5b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/de43a640083c49b19a197347ea969a5b 2023-03-08 11:55:40,325 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 1b2090f00e0a2d838e28358e6d6297e4/family1 of 1b2090f00e0a2d838e28358e6d6297e4 into de43a640083c49b19a197347ea969a5b(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:55:40,325 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:40,325 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678276539395; duration=0sec 2023-03-08 11:55:40,325 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-08 11:55:40,326 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 11:55:40,327 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 11:55:40,327 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1912): 8a2814671292e9b2760956bf02135606/family2 is initiating minor compaction (all files) 2023-03-08 11:55:40,327 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8a2814671292e9b2760956bf02135606/family2 in testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:55:40,327 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-9c92d16d2b9f43529ed6b665e31e6836, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-3a0c4db82b484bbea43a20cc9970f523, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-855e4339fb014cda9a4a8621eb2c9cc5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-5a3c6c586c1d4a2da00c3fe70280debf] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp, totalSize=20.4 K 2023-03-08 11:55:40,328 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-9c92d16d2b9f43529ed6b665e31e6836, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514611 2023-03-08 11:55:40,328 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-3a0c4db82b484bbea43a20cc9970f523, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516445 2023-03-08 11:55:40,329 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-855e4339fb014cda9a4a8621eb2c9cc5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518239 2023-03-08 11:55:40,329 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-5a3c6c586c1d4a2da00c3fe70280debf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:40,340 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8a2814671292e9b2760956bf02135606#family2#compaction#116 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:40,373 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3356b1ae0f4c4f8996df9885e552ca20, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-dbcdd7d7d25445c094214456a69dcade, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-bc161e308bab4d49a0b696e98fc1727c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-4183389884c8402d912826df5bd5dbfe, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/a478acbc799944c8afb63b82457fe276_SeqId_34_] to archive 2023-03-08 11:55:40,373 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:40,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3356b1ae0f4c4f8996df9885e552ca20 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3356b1ae0f4c4f8996df9885e552ca20 2023-03-08 11:55:40,376 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-dbcdd7d7d25445c094214456a69dcade to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-dbcdd7d7d25445c094214456a69dcade 2023-03-08 11:55:40,376 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-bc161e308bab4d49a0b696e98fc1727c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-bc161e308bab4d49a0b696e98fc1727c 2023-03-08 11:55:40,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-4183389884c8402d912826df5bd5dbfe to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-4183389884c8402d912826df5bd5dbfe 2023-03-08 11:55:40,378 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/a478acbc799944c8afb63b82457fe276_SeqId_34_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family1/a478acbc799944c8afb63b82457fe276_SeqId_34_ 2023-03-08 11:55:40,389 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:40,389 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 64864abb7d0098b010ede90e39027ce9, disabling compactions & flushes 2023-03-08 11:55:40,389 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:40,389 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:40,389 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. after waiting 0 ms 2023-03-08 11:55:40,389 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:40,390 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:55:40,390 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:40,390 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:55:40,392 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:55:40,393 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678276540392"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276540392"}]},"ts":"1678276540392"} 2023-03-08 11:55:40,393 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678276540392"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276540392"}]},"ts":"1678276540392"} 2023-03-08 11:55:40,393 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678276540392"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276540392"}]},"ts":"1678276540392"} 2023-03-08 11:55:40,395 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 11:55:40,396 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:55:40,396 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276540396"}]},"ts":"1678276540396"} 2023-03-08 11:55:40,397 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-08 11:55:40,400 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6249f9a64ffef0dbc4b217b148070179, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b617fde530c0c1bebf3fa2fa6eab77bb, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=64864abb7d0098b010ede90e39027ce9, ASSIGN}] 2023-03-08 11:55:40,401 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6249f9a64ffef0dbc4b217b148070179, ASSIGN 2023-03-08 11:55:40,402 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b617fde530c0c1bebf3fa2fa6eab77bb, ASSIGN 2023-03-08 11:55:40,402 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=64864abb7d0098b010ede90e39027ce9, ASSIGN 2023-03-08 11:55:40,402 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6249f9a64ffef0dbc4b217b148070179, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:55:40,402 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b617fde530c0c1bebf3fa2fa6eab77bb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:55:40,402 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=64864abb7d0098b010ede90e39027ce9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:55:40,555 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=b617fde530c0c1bebf3fa2fa6eab77bb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:40,555 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=6249f9a64ffef0dbc4b217b148070179, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:40,555 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=64864abb7d0098b010ede90e39027ce9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:40,555 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678276540555"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276540555"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276540555"}]},"ts":"1678276540555"} 2023-03-08 11:55:40,555 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678276540555"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276540555"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276540555"}]},"ts":"1678276540555"} 2023-03-08 11:55:40,555 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678276540555"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276540555"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276540555"}]},"ts":"1678276540555"} 2023-03-08 11:55:40,557 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure b617fde530c0c1bebf3fa2fa6eab77bb, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:40,557 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=108, state=RUNNABLE; OpenRegionProcedure 64864abb7d0098b010ede90e39027ce9, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:40,558 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=106, state=RUNNABLE; OpenRegionProcedure 6249f9a64ffef0dbc4b217b148070179, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:55:40,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 11:55:40,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:40,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 64864abb7d0098b010ede90e39027ce9, NAME => 'ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 11:55:40,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 64864abb7d0098b010ede90e39027ce9 2023-03-08 11:55:40,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:40,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 64864abb7d0098b010ede90e39027ce9 2023-03-08 11:55:40,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 64864abb7d0098b010ede90e39027ce9 2023-03-08 11:55:40,712 INFO [StoreOpener-64864abb7d0098b010ede90e39027ce9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 64864abb7d0098b010ede90e39027ce9 2023-03-08 11:55:40,714 DEBUG [StoreOpener-64864abb7d0098b010ede90e39027ce9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1 2023-03-08 11:55:40,714 DEBUG [StoreOpener-64864abb7d0098b010ede90e39027ce9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1 2023-03-08 11:55:40,714 INFO [StoreOpener-64864abb7d0098b010ede90e39027ce9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 64864abb7d0098b010ede90e39027ce9 columnFamilyName family1 2023-03-08 11:55:40,715 INFO [StoreOpener-64864abb7d0098b010ede90e39027ce9-1] regionserver.HStore(310): Store=64864abb7d0098b010ede90e39027ce9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:40,716 INFO [StoreOpener-64864abb7d0098b010ede90e39027ce9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 64864abb7d0098b010ede90e39027ce9 2023-03-08 11:55:40,717 DEBUG [StoreOpener-64864abb7d0098b010ede90e39027ce9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2 2023-03-08 11:55:40,717 DEBUG [StoreOpener-64864abb7d0098b010ede90e39027ce9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2 2023-03-08 11:55:40,717 INFO [StoreOpener-64864abb7d0098b010ede90e39027ce9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 64864abb7d0098b010ede90e39027ce9 columnFamilyName family2 2023-03-08 11:55:40,719 INFO [StoreOpener-64864abb7d0098b010ede90e39027ce9-1] regionserver.HStore(310): Store=64864abb7d0098b010ede90e39027ce9/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:40,719 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family1/e10ba531265b454197fc31a289235070 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/e10ba531265b454197fc31a289235070 2023-03-08 11:55:40,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9 2023-03-08 11:55:40,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9 2023-03-08 11:55:40,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:55:40,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 64864abb7d0098b010ede90e39027ce9 2023-03-08 11:55:40,723 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 8a2814671292e9b2760956bf02135606/family1 of 8a2814671292e9b2760956bf02135606 into e10ba531265b454197fc31a289235070(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:40,723 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:40,723 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678276539402; duration=1sec 2023-03-08 11:55:40,723 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-08 11:55:40,724 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 11:55:40,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:55:40,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 64864abb7d0098b010ede90e39027ce9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64595015, jitterRate=-0.03745926916599274}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:55:40,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:55:40,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9., pid=110, masterSystemTime=1678276540708 2023-03-08 11:55:40,726 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 11:55:40,726 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HStore(1912): 1b2090f00e0a2d838e28358e6d6297e4/family2 is initiating minor compaction (all files) 2023-03-08 11:55:40,726 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1b2090f00e0a2d838e28358e6d6297e4/family2 in testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:55:40,726 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-9d982a00f7234f3799eff4331f7781ea, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-b4ac784cd164469a8b3459a552bab0db, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3e8179efb3d04e28ab09e2d11966c946, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-5908c3bffaf24fcaa736f19e125a250b] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp, totalSize=20.4 K 2023-03-08 11:55:40,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:40,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:40,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:40,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b617fde530c0c1bebf3fa2fa6eab77bb, NAME => 'ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 11:55:40,727 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=64864abb7d0098b010ede90e39027ce9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:40,727 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-9d982a00f7234f3799eff4331f7781ea, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276514610 2023-03-08 11:55:40,727 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678276540727"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276540727"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276540727"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276540727"}]},"ts":"1678276540727"} 2023-03-08 11:55:40,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:55:40,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:40,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:55:40,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:55:40,728 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-b4ac784cd164469a8b3459a552bab0db, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276516445 2023-03-08 11:55:40,729 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3e8179efb3d04e28ab09e2d11966c946, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276518238 2023-03-08 11:55:40,729 INFO [StoreOpener-b617fde530c0c1bebf3fa2fa6eab77bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:55:40,729 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-5908c3bffaf24fcaa736f19e125a250b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276520035 2023-03-08 11:55:40,730 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=108 2023-03-08 11:55:40,730 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=108, state=SUCCESS; OpenRegionProcedure 64864abb7d0098b010ede90e39027ce9, server=jenkins-hbase4.apache.org,45551,1678276414946 in 171 msec 2023-03-08 11:55:40,730 DEBUG [StoreOpener-b617fde530c0c1bebf3fa2fa6eab77bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1 2023-03-08 11:55:40,731 DEBUG [StoreOpener-b617fde530c0c1bebf3fa2fa6eab77bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1 2023-03-08 11:55:40,731 INFO [StoreOpener-b617fde530c0c1bebf3fa2fa6eab77bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b617fde530c0c1bebf3fa2fa6eab77bb columnFamilyName family1 2023-03-08 11:55:40,731 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=64864abb7d0098b010ede90e39027ce9, ASSIGN in 330 msec 2023-03-08 11:55:40,732 INFO [StoreOpener-b617fde530c0c1bebf3fa2fa6eab77bb-1] regionserver.HStore(310): Store=b617fde530c0c1bebf3fa2fa6eab77bb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:40,732 INFO [StoreOpener-b617fde530c0c1bebf3fa2fa6eab77bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:55:40,734 DEBUG [StoreOpener-b617fde530c0c1bebf3fa2fa6eab77bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2 2023-03-08 11:55:40,734 DEBUG [StoreOpener-b617fde530c0c1bebf3fa2fa6eab77bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2 2023-03-08 11:55:40,734 INFO [StoreOpener-b617fde530c0c1bebf3fa2fa6eab77bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b617fde530c0c1bebf3fa2fa6eab77bb columnFamilyName family2 2023-03-08 11:55:40,735 INFO [StoreOpener-b617fde530c0c1bebf3fa2fa6eab77bb-1] regionserver.HStore(310): Store=b617fde530c0c1bebf3fa2fa6eab77bb/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:40,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:55:40,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:55:40,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:55:40,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:55:40,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:55:40,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b617fde530c0c1bebf3fa2fa6eab77bb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59447462, jitterRate=-0.11416378617286682}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:55:40,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:55:40,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb., pid=109, masterSystemTime=1678276540708 2023-03-08 11:55:40,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:40,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:40,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:40,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6249f9a64ffef0dbc4b217b148070179, NAME => 'ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 11:55:40,745 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=b617fde530c0c1bebf3fa2fa6eab77bb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:40,745 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] throttle.PressureAwareThroughputController(145): 1b2090f00e0a2d838e28358e6d6297e4#family2#compaction#117 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:40,745 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678276540745"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276540745"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276540745"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276540745"}]},"ts":"1678276540745"} 2023-03-08 11:55:40,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:55:40,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:40,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:55:40,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:55:40,747 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=107 2023-03-08 11:55:40,747 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure b617fde530c0c1bebf3fa2fa6eab77bb, server=jenkins-hbase4.apache.org,45551,1678276414946 in 189 msec 2023-03-08 11:55:40,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b617fde530c0c1bebf3fa2fa6eab77bb, ASSIGN in 347 msec 2023-03-08 11:55:40,749 INFO [StoreOpener-6249f9a64ffef0dbc4b217b148070179-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:55:40,752 DEBUG [StoreOpener-6249f9a64ffef0dbc4b217b148070179-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1 2023-03-08 11:55:40,752 DEBUG [StoreOpener-6249f9a64ffef0dbc4b217b148070179-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1 2023-03-08 11:55:40,752 INFO [StoreOpener-6249f9a64ffef0dbc4b217b148070179-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6249f9a64ffef0dbc4b217b148070179 columnFamilyName family1 2023-03-08 11:55:40,753 INFO [StoreOpener-6249f9a64ffef0dbc4b217b148070179-1] regionserver.HStore(310): Store=6249f9a64ffef0dbc4b217b148070179/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:40,754 INFO [StoreOpener-6249f9a64ffef0dbc4b217b148070179-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:55:40,755 DEBUG [StoreOpener-6249f9a64ffef0dbc4b217b148070179-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2 2023-03-08 11:55:40,755 DEBUG [StoreOpener-6249f9a64ffef0dbc4b217b148070179-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2 2023-03-08 11:55:40,755 INFO [StoreOpener-6249f9a64ffef0dbc4b217b148070179-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6249f9a64ffef0dbc4b217b148070179 columnFamilyName family2 2023-03-08 11:55:40,756 INFO [StoreOpener-6249f9a64ffef0dbc4b217b148070179-1] regionserver.HStore(310): Store=6249f9a64ffef0dbc4b217b148070179/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:55:40,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:55:40,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:55:40,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:55:40,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:55:40,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:55:40,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6249f9a64ffef0dbc4b217b148070179; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63647908, jitterRate=-0.05157226324081421}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:55:40,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:55:40,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179., pid=111, masterSystemTime=1678276540708 2023-03-08 11:55:40,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:40,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:40,769 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=6249f9a64ffef0dbc4b217b148070179, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:40,769 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678276540769"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276540769"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276540769"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276540769"}]},"ts":"1678276540769"} 2023-03-08 11:55:40,771 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=106 2023-03-08 11:55:40,771 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=106, state=SUCCESS; OpenRegionProcedure 6249f9a64ffef0dbc4b217b148070179, server=jenkins-hbase4.apache.org,45551,1678276414946 in 212 msec 2023-03-08 11:55:40,773 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-03-08 11:55:40,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-12fb73322f2b462daa9f7f2b5e2177a2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-051d4a6cbfe34e7a96dc643c2b20cf3a, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-1cd5f0aa679d4c86b010d1eb4ea4235c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-e99f9377a8d64fbf9813721496bc314c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/eaf94b0f8129492cae1b45247aa8d7a5_SeqId_34_] to archive 2023-03-08 11:55:40,773 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6249f9a64ffef0dbc4b217b148070179, ASSIGN in 371 msec 2023-03-08 11:55:40,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:40,773 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:55:40,773 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276540773"}]},"ts":"1678276540773"} 2023-03-08 11:55:40,774 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-08 11:55:40,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-12fb73322f2b462daa9f7f2b5e2177a2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-12fb73322f2b462daa9f7f2b5e2177a2 2023-03-08 11:55:40,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-051d4a6cbfe34e7a96dc643c2b20cf3a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-051d4a6cbfe34e7a96dc643c2b20cf3a 2023-03-08 11:55:40,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-1cd5f0aa679d4c86b010d1eb4ea4235c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-1cd5f0aa679d4c86b010d1eb4ea4235c 2023-03-08 11:55:40,777 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:55:40,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-e99f9377a8d64fbf9813721496bc314c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-e99f9377a8d64fbf9813721496bc314c 2023-03-08 11:55:40,778 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2100 sec 2023-03-08 11:55:40,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/eaf94b0f8129492cae1b45247aa8d7a5_SeqId_34_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family1/eaf94b0f8129492cae1b45247aa8d7a5_SeqId_34_ 2023-03-08 11:55:41,169 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/.tmp/family2/1f929f9b91e84e309b49144b653e4ccd as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/1f929f9b91e84e309b49144b653e4ccd 2023-03-08 11:55:41,173 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 8a2814671292e9b2760956bf02135606/family2 of 8a2814671292e9b2760956bf02135606 into 1f929f9b91e84e309b49144b653e4ccd(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:55:41,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:55:41,173 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606., storeName=8a2814671292e9b2760956bf02135606/family2, priority=12, startTime=1678276538288; duration=0sec 2023-03-08 11:55:41,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-08 11:55:41,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-08 11:55:41,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 11:55:41,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 11:55:41,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. because compaction request was cancelled 2023-03-08 11:55:41,173 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-08 11:55:41,174 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 11:55:41,174 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 11:55:41,174 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. because compaction request was cancelled 2023-03-08 11:55:41,273 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-9c92d16d2b9f43529ed6b665e31e6836, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-3a0c4db82b484bbea43a20cc9970f523, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-855e4339fb014cda9a4a8621eb2c9cc5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-5a3c6c586c1d4a2da00c3fe70280debf] to archive 2023-03-08 11:55:41,274 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:41,275 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-9c92d16d2b9f43529ed6b665e31e6836 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-9c92d16d2b9f43529ed6b665e31e6836 2023-03-08 11:55:41,276 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-3a0c4db82b484bbea43a20cc9970f523 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-3a0c4db82b484bbea43a20cc9970f523 2023-03-08 11:55:41,277 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-855e4339fb014cda9a4a8621eb2c9cc5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-855e4339fb014cda9a4a8621eb2c9cc5 2023-03-08 11:55:41,278 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-5a3c6c586c1d4a2da00c3fe70280debf to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/family2/testCompactionWithHFileLink=8a2814671292e9b2760956bf02135606-5a3c6c586c1d4a2da00c3fe70280debf 2023-03-08 11:55:41,573 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/.tmp/family2/acdbafe7e7674ccebdccded606c15e0a as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/acdbafe7e7674ccebdccded606c15e0a 2023-03-08 11:55:41,577 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 1b2090f00e0a2d838e28358e6d6297e4/family2 of 1b2090f00e0a2d838e28358e6d6297e4 into acdbafe7e7674ccebdccded606c15e0a(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:55:41,577 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:55:41,577 INFO [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4., storeName=1b2090f00e0a2d838e28358e6d6297e4/family2, priority=12, startTime=1678276538367; duration=0sec 2023-03-08 11:55:41,577 DEBUG [RS:0;jenkins-hbase4:45551-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:55:41,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-9d982a00f7234f3799eff4331f7781ea, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-b4ac784cd164469a8b3459a552bab0db, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3e8179efb3d04e28ab09e2d11966c946, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-5908c3bffaf24fcaa736f19e125a250b] to archive 2023-03-08 11:55:41,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:41,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 11:55:41,675 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-08 11:55:41,675 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-9d982a00f7234f3799eff4331f7781ea to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-9d982a00f7234f3799eff4331f7781ea 2023-03-08 11:55:41,676 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-b4ac784cd164469a8b3459a552bab0db to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-b4ac784cd164469a8b3459a552bab0db 2023-03-08 11:55:41,677 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3e8179efb3d04e28ab09e2d11966c946 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-3e8179efb3d04e28ab09e2d11966c946 2023-03-08 11:55:41,679 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-5908c3bffaf24fcaa736f19e125a250b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/family2/testCompactionWithHFileLink=1b2090f00e0a2d838e28358e6d6297e4-5908c3bffaf24fcaa736f19e125a250b 2023-03-08 11:55:41,681 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:41,681 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:41,681 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:41,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:41,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 11:55:41,684 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 11:55:41,685 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:41,685 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 11:55:41,685 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 11:55:41,685 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:41,685 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:41,686 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:41,686 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:41,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:41,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:41,686 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:41,686 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:41,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:41,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:41,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:41,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:41,687 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 11:55:41,687 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 11:55:41,687 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 11:55:41,687 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:41,687 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 11:55:41,688 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:41,688 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. started... 2023-03-08 11:55:41,688 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:41,688 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. started... 2023-03-08 11:55:41,689 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:55:41,689 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 64864abb7d0098b010ede90e39027ce9 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 11:55:41,690 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b617fde530c0c1bebf3fa2fa6eab77bb 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 11:55:41,690 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:41,689 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:41,690 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. started... 2023-03-08 11:55:41,690 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6249f9a64ffef0dbc4b217b148070179 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 11:55:42,105 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308c171e2359512498297f58232adc70e02 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308c171e2359512498297f58232adc70e02 2023-03-08 11:55:42,105 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308ec0edea7c87442c1ab175026ad017076 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308ec0edea7c87442c1ab175026ad017076 2023-03-08 11:55:42,106 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308ccbc1fe6515e49c9a890703301b43830 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308ccbc1fe6515e49c9a890703301b43830 2023-03-08 11:55:42,522 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/be342fae247642e2b85432ca186c01b2 2023-03-08 11:55:42,522 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/2cd44157d98849529cfbf008d6bfd7ef 2023-03-08 11:55:42,522 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/3f38423a484a464591de987cabbc5e59 2023-03-08 11:55:42,544 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-08 11:55:42,949 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308656148af8c0547b9b6e83bec2504031a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230308656148af8c0547b9b6e83bec2504031a 2023-03-08 11:55:42,950 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030883a90bf5befe44d4be02eef69359e31d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023030883a90bf5befe44d4be02eef69359e31d 2023-03-08 11:55:42,950 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303081aea5205e9c94126a24f7a9c1bb771ff to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303081aea5205e9c94126a24f7a9c1bb771ff 2023-03-08 11:55:43,354 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/d5cc3f3ddcdf4c14ab934a7b682577ea 2023-03-08 11:55:43,358 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/3f38423a484a464591de987cabbc5e59 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/3f38423a484a464591de987cabbc5e59 2023-03-08 11:55:43,361 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/3f38423a484a464591de987cabbc5e59, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 11:55:43,362 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/efb5dae7e2f746a8a5a56dd52fc4a4dc 2023-03-08 11:55:43,362 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/25514da1a31145e98f50eaf5ee044b18 2023-03-08 11:55:43,362 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/d5cc3f3ddcdf4c14ab934a7b682577ea as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/d5cc3f3ddcdf4c14ab934a7b682577ea 2023-03-08 11:55:43,365 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/2cd44157d98849529cfbf008d6bfd7ef as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/2cd44157d98849529cfbf008d6bfd7ef 2023-03-08 11:55:43,366 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/d5cc3f3ddcdf4c14ab934a7b682577ea, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 11:55:43,366 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/be342fae247642e2b85432ca186c01b2 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/be342fae247642e2b85432ca186c01b2 2023-03-08 11:55:43,367 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6249f9a64ffef0dbc4b217b148070179 in 1677ms, sequenceid=5, compaction requested=false 2023-03-08 11:55:43,367 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:55:43,367 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:43,367 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:55:43,370 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/2cd44157d98849529cfbf008d6bfd7ef, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 11:55:43,370 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/efb5dae7e2f746a8a5a56dd52fc4a4dc as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/efb5dae7e2f746a8a5a56dd52fc4a4dc 2023-03-08 11:55:43,373 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/efb5dae7e2f746a8a5a56dd52fc4a4dc, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 11:55:43,374 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for b617fde530c0c1bebf3fa2fa6eab77bb in 1685ms, sequenceid=5, compaction requested=false 2023-03-08 11:55:43,374 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:55:43,374 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:43,374 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:55:43,376 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/be342fae247642e2b85432ca186c01b2, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 11:55:43,377 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/25514da1a31145e98f50eaf5ee044b18 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/25514da1a31145e98f50eaf5ee044b18 2023-03-08 11:55:43,380 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/25514da1a31145e98f50eaf5ee044b18, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 11:55:43,380 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 64864abb7d0098b010ede90e39027ce9 in 1691ms, sequenceid=5, compaction requested=false 2023-03-08 11:55:43,380 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:55:43,380 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:43,380 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:55:43,381 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:55:43,381 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,381 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 11:55:43,381 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 11:55:43,382 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,382 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:43,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:43,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:43,382 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:43,382 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:43,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:43,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:43,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:43,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:43,384 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 11:55:43,384 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57eaa0eb[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:43,384 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 11:55:43,384 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:43,385 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:43,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:43,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:43,385 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 11:55:43,385 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 11:55:43,385 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:43,385 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,385 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:43,388 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,388 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 11:55:43,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:43,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:43,388 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:43,388 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 11:55:43,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:43,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:43,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:43,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:43,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:43,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:43,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 11:55:43,390 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 11:55:43,390 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:43,390 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:43,390 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 11:55:43,390 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:43,392 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:43,392 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:43,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:43,392 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:43,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:43,392 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:43,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:43,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:43,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,393 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:43,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:43,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,395 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,395 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:43,395 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:43,395 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:43,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 11:55:43,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:43,395 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:43,395 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:43,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:43,396 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 11:55:43,396 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:43,396 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:43,396 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:43,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:43,396 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:43,396 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:43,496 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:43,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:43,498 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:43,498 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:43,498 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:43,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:43,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 11:55:43,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 11:55:43,501 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 11:55:43,501 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:43,501 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 11:55:43,502 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 11:55:43,502 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:43,502 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:43,503 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,503 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:43,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:43,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:43,503 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:43,503 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:43,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:43,504 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:43,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:43,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:43,504 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 11:55:43,504 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 11:55:43,504 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 11:55:43,504 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 11:55:43,504 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:43,504 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 11:55:43,505 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:43,505 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:55:43,505 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:43,505 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:43,505 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. started... 2023-03-08 11:55:43,505 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:43,505 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. started... 2023-03-08 11:55:43,505 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. started... 2023-03-08 11:55:43,505 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6249f9a64ffef0dbc4b217b148070179 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 11:55:43,505 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b617fde530c0c1bebf3fa2fa6eab77bb 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 11:55:43,505 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 64864abb7d0098b010ede90e39027ce9 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 11:55:43,921 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303087a65b82663b44898a1e9e3f03a66bb5a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303087a65b82663b44898a1e9e3f03a66bb5a 2023-03-08 11:55:43,922 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308f90abf95b901495dbe97801b03c3ff45 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308f90abf95b901495dbe97801b03c3ff45 2023-03-08 11:55:43,922 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308570d70740eea48aea60adf76cd99e07b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308570d70740eea48aea60adf76cd99e07b 2023-03-08 11:55:44,328 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/2bc6fe67a3e2487d856842de3f07f2e6 2023-03-08 11:55:44,329 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/afc4ddbf11684bcab57844e45b7bc7fe 2023-03-08 11:55:44,329 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/2c11019d72a6400a853fed707ae43e83 2023-03-08 11:55:44,743 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308f5838f0d7e024c47b80fdbcbd20f0a96 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230308f5838f0d7e024c47b80fdbcbd20f0a96 2023-03-08 11:55:44,743 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303085fca62f5966a477a9998061350ef5694 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303085fca62f5966a477a9998061350ef5694 2023-03-08 11:55:44,743 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303087569e33b509548b3a2034902ec9dc741 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303087569e33b509548b3a2034902ec9dc741 2023-03-08 11:55:45,148 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/51c55a53b9c94221bf1b0d5f5932aae5 2023-03-08 11:55:45,150 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/ee5dc82f18f84556a736f46877971086 2023-03-08 11:55:45,150 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/da7bdf919dc84f03a829a85bc7d26f91 2023-03-08 11:55:45,152 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/2c11019d72a6400a853fed707ae43e83 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/2c11019d72a6400a853fed707ae43e83 2023-03-08 11:55:45,155 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/2bc6fe67a3e2487d856842de3f07f2e6 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2bc6fe67a3e2487d856842de3f07f2e6 2023-03-08 11:55:45,155 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/afc4ddbf11684bcab57844e45b7bc7fe as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/afc4ddbf11684bcab57844e45b7bc7fe 2023-03-08 11:55:45,157 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/2c11019d72a6400a853fed707ae43e83, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 11:55:45,158 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/51c55a53b9c94221bf1b0d5f5932aae5 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/51c55a53b9c94221bf1b0d5f5932aae5 2023-03-08 11:55:45,158 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2bc6fe67a3e2487d856842de3f07f2e6, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 11:55:45,159 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/afc4ddbf11684bcab57844e45b7bc7fe, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 11:55:45,159 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/ee5dc82f18f84556a736f46877971086 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/ee5dc82f18f84556a736f46877971086 2023-03-08 11:55:45,159 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/da7bdf919dc84f03a829a85bc7d26f91 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/da7bdf919dc84f03a829a85bc7d26f91 2023-03-08 11:55:45,162 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/51c55a53b9c94221bf1b0d5f5932aae5, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 11:55:45,163 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/ee5dc82f18f84556a736f46877971086, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 11:55:45,163 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6249f9a64ffef0dbc4b217b148070179 in 1658ms, sequenceid=9, compaction requested=false 2023-03-08 11:55:45,163 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/da7bdf919dc84f03a829a85bc7d26f91, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 11:55:45,163 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:55:45,163 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:45,163 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:55:45,163 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 64864abb7d0098b010ede90e39027ce9 in 1658ms, sequenceid=9, compaction requested=false 2023-03-08 11:55:45,164 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:55:45,164 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:45,164 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:55:45,164 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for b617fde530c0c1bebf3fa2fa6eab77bb in 1659ms, sequenceid=9, compaction requested=false 2023-03-08 11:55:45,164 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:55:45,164 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:45,164 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:55:45,164 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:55:45,164 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,164 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 11:55:45,164 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 11:55:45,167 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,167 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:45,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:45,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:45,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:45,167 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:45,167 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:45,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:45,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:45,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:45,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 11:55:45,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cc3237a[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:45,169 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 11:55:45,169 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:45,170 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:45,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:45,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:45,171 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 11:55:45,171 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 11:55:45,171 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:45,171 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,171 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:45,175 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,175 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 11:55:45,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:45,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:45,175 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:45,175 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 11:55:45,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:45,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:45,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:45,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:45,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:45,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:45,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 11:55:45,178 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 11:55:45,178 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:45,178 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:45,178 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 11:55:45,178 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:45,180 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:45,180 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:45,180 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:45,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:45,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:45,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:45,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:45,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:45,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:45,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:45,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:45,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:45,181 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:45,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:45,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:45,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,182 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:45,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:45,184 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:45,185 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,185 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:45,185 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:45,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 11:55:45,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:45,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:45,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:45,185 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:45,185 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:45,185 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:45,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:45,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 11:55:45,185 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:45,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:45,185 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:45,286 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:45,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:45,288 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:45,288 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:45,288 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:45,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:45,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 11:55:45,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 11:55:45,292 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 11:55:45,292 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:45,292 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 11:55:45,292 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 11:55:45,293 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:45,293 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:45,294 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,294 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:45,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:45,294 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:45,294 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:45,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:45,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 11:55:45,295 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 11:55:45,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 11:55:45,295 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 11:55:45,295 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:45,295 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 11:55:45,296 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:45,296 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. started... 2023-03-08 11:55:45,296 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:45,296 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 64864abb7d0098b010ede90e39027ce9 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 11:55:45,296 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. started... 2023-03-08 11:55:45,296 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:45,296 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. started... 2023-03-08 11:55:45,296 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:55:45,296 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6249f9a64ffef0dbc4b217b148070179 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 11:55:45,296 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b617fde530c0c1bebf3fa2fa6eab77bb 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 11:55:45,296 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:45,715 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030862f2fad75ad9458996cdc1b70af5265c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023030862f2fad75ad9458996cdc1b70af5265c 2023-03-08 11:55:45,715 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030807eac3dc11bf4ba5a1ecf49104ddef8e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023030807eac3dc11bf4ba5a1ecf49104ddef8e 2023-03-08 11:55:45,715 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030805d0ad123a2745248dfe99dbaddb9309 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023030805d0ad123a2745248dfe99dbaddb9309 2023-03-08 11:55:45,730 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/e5e3731fb27c43f2a7b7f89cf5a43e63 2023-03-08 11:55:45,742 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030817daa9f4786946feb63107e8d2c5db3c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023030817daa9f4786946feb63107e8d2c5db3c 2023-03-08 11:55:46,129 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/dca727f6da6b4ada8158f6d219ffd2d6 2023-03-08 11:55:46,130 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/98785f26ff904586a17880243dcbfd5f 2023-03-08 11:55:46,146 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/643e438faed94d898daee210d4232d9d 2023-03-08 11:55:46,150 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/e5e3731fb27c43f2a7b7f89cf5a43e63 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/e5e3731fb27c43f2a7b7f89cf5a43e63 2023-03-08 11:55:46,154 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/e5e3731fb27c43f2a7b7f89cf5a43e63, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 11:55:46,154 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/643e438faed94d898daee210d4232d9d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/643e438faed94d898daee210d4232d9d 2023-03-08 11:55:46,157 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/643e438faed94d898daee210d4232d9d, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 11:55:46,158 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 64864abb7d0098b010ede90e39027ce9 in 862ms, sequenceid=13, compaction requested=true 2023-03-08 11:55:46,158 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:55:46,158 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:46,159 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:55:46,213 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:55:46,542 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303083c13c5c9f90648d896171701656d77db to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303083c13c5c9f90648d896171701656d77db 2023-03-08 11:55:46,549 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303083a13d79f5aa44b3492e57e6c7b555816 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303083a13d79f5aa44b3492e57e6c7b555816 2023-03-08 11:55:46,947 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/340d9effbdfa40d2b5e2f84752eb03ec 2023-03-08 11:55:46,951 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/dca727f6da6b4ada8158f6d219ffd2d6 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/dca727f6da6b4ada8158f6d219ffd2d6 2023-03-08 11:55:46,953 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/81effd51b5354aaca70d4b9e1f5ff66d 2023-03-08 11:55:46,956 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/dca727f6da6b4ada8158f6d219ffd2d6, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 11:55:46,956 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/340d9effbdfa40d2b5e2f84752eb03ec as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/340d9effbdfa40d2b5e2f84752eb03ec 2023-03-08 11:55:46,957 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/98785f26ff904586a17880243dcbfd5f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/98785f26ff904586a17880243dcbfd5f 2023-03-08 11:55:46,960 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/340d9effbdfa40d2b5e2f84752eb03ec, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 11:55:46,960 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/98785f26ff904586a17880243dcbfd5f, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 11:55:46,961 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/81effd51b5354aaca70d4b9e1f5ff66d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/81effd51b5354aaca70d4b9e1f5ff66d 2023-03-08 11:55:46,961 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for b617fde530c0c1bebf3fa2fa6eab77bb in 1665ms, sequenceid=13, compaction requested=true 2023-03-08 11:55:46,961 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:55:46,961 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:46,961 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:55:46,965 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/81effd51b5354aaca70d4b9e1f5ff66d, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 11:55:46,965 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6249f9a64ffef0dbc4b217b148070179 in 1669ms, sequenceid=13, compaction requested=true 2023-03-08 11:55:46,966 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:55:46,966 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:46,966 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:55:46,966 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:55:46,966 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,966 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 11:55:46,966 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 11:55:46,967 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:46,967 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:46,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:46,968 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:46,968 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:46,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 11:55:46,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e2c31f1[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:46,969 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 11:55:46,969 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:46,971 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:46,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:46,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:46,971 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 11:55:46,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 11:55:46,971 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:46,971 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:46,972 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,972 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 11:55:46,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:46,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:46,972 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:46,973 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 11:55:46,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:46,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:46,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:46,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:46,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:46,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:46,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 11:55:46,974 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 11:55:46,974 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:46,974 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:46,974 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 11:55:46,974 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:46,976 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:46,976 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:46,976 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:46,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:46,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:46,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:46,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:46,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:46,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:46,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:46,976 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:46,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:46,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:46,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:46,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:46,977 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:46,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:46,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,979 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,979 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:46,979 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:46,979 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:46,979 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:46,979 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:46,979 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:46,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 11:55:46,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:46,979 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:46,979 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:46,979 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:46,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:46,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:46,979 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:46,980 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:46,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:46,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:47,080 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:47,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:47,082 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:47,082 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:47,082 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:55:47,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:47,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 11:55:47,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 11:55:47,086 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 11:55:47,086 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:47,086 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 11:55:47,086 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 11:55:47,086 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:47,086 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:47,089 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:47,089 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:47,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:47,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:47,089 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:47,089 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:47,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:47,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:47,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 11:55:47,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 11:55:47,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 11:55:47,090 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 11:55:47,090 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:47,090 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 11:55:47,090 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:47,090 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:55:47,090 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:47,090 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. started... 2023-03-08 11:55:47,090 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:47,090 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:47,091 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 64864abb7d0098b010ede90e39027ce9 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 11:55:47,091 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. started... 2023-03-08 11:55:47,091 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. started... 2023-03-08 11:55:47,091 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b617fde530c0c1bebf3fa2fa6eab77bb 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 11:55:47,091 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6249f9a64ffef0dbc4b217b148070179 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 11:55:47,104 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030883d453da6f8f4f44bfb5ee3ee7ca12f5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023030883d453da6f8f4f44bfb5ee3ee7ca12f5 2023-03-08 11:55:47,109 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/beea862a0d664985af3914d0e74f5eb2 2023-03-08 11:55:47,503 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308d0cc2ffa381a4fe9b092ffd7fc6ca42c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308d0cc2ffa381a4fe9b092ffd7fc6ca42c 2023-03-08 11:55:47,505 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308151d18b7e92145deb6828d678fcecfde to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308151d18b7e92145deb6828d678fcecfde 2023-03-08 11:55:47,520 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030804a189abf3bf47eaa4935be65599b112 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023030804a189abf3bf47eaa4935be65599b112 2023-03-08 11:55:47,908 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/94c459887801443aafb04ca3498ac6d1 2023-03-08 11:55:47,909 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/2d3bbaff12904dea8f7bfc28b175675c 2023-03-08 11:55:47,924 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/109e80bc9feb4bf283653bc2b6a2e5a7 2023-03-08 11:55:47,927 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/beea862a0d664985af3914d0e74f5eb2 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/beea862a0d664985af3914d0e74f5eb2 2023-03-08 11:55:47,930 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/beea862a0d664985af3914d0e74f5eb2, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 11:55:47,931 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/109e80bc9feb4bf283653bc2b6a2e5a7 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/109e80bc9feb4bf283653bc2b6a2e5a7 2023-03-08 11:55:47,934 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/109e80bc9feb4bf283653bc2b6a2e5a7, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 11:55:47,934 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for b617fde530c0c1bebf3fa2fa6eab77bb in 843ms, sequenceid=17, compaction requested=true 2023-03-08 11:55:47,934 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:55:47,934 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:47,934 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:55:48,322 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303088784688d74e84a4ba00df8bc2a9016fe to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303088784688d74e84a4ba00df8bc2a9016fe 2023-03-08 11:55:48,322 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308b18d1e91ff8f463daa1fca2a171f8de9 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230308b18d1e91ff8f463daa1fca2a171f8de9 2023-03-08 11:55:48,727 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/567b9d336aff4f1885bdbd8c4a6ef933 2023-03-08 11:55:48,727 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/06914855ae444a38b3b4728b06307e8d 2023-03-08 11:55:48,731 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/94c459887801443aafb04ca3498ac6d1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/94c459887801443aafb04ca3498ac6d1 2023-03-08 11:55:48,731 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/2d3bbaff12904dea8f7bfc28b175675c as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2d3bbaff12904dea8f7bfc28b175675c 2023-03-08 11:55:48,734 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/94c459887801443aafb04ca3498ac6d1, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 11:55:48,734 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2d3bbaff12904dea8f7bfc28b175675c, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 11:55:48,735 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/567b9d336aff4f1885bdbd8c4a6ef933 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/567b9d336aff4f1885bdbd8c4a6ef933 2023-03-08 11:55:48,735 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/06914855ae444a38b3b4728b06307e8d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/06914855ae444a38b3b4728b06307e8d 2023-03-08 11:55:48,738 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/567b9d336aff4f1885bdbd8c4a6ef933, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 11:55:48,739 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/06914855ae444a38b3b4728b06307e8d, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 11:55:48,739 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6249f9a64ffef0dbc4b217b148070179 in 1648ms, sequenceid=17, compaction requested=true 2023-03-08 11:55:48,739 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 64864abb7d0098b010ede90e39027ce9 in 1649ms, sequenceid=17, compaction requested=true 2023-03-08 11:55:48,739 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:55:48,739 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:55:48,739 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:48,739 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:48,739 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:55:48,739 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:55:48,739 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:55:48,739 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,739 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 11:55:48,739 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 11:55:48,741 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:48,741 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:48,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:48,741 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:48,741 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:48,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:48,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:48,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:48,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:48,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 11:55:48,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b184e5c[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:48,742 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 11:55:48,742 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:48,743 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:48,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:48,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:48,743 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,743 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 11:55:48,744 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 11:55:48,743 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:48,744 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:48,745 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,745 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 11:55:48,745 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:48,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,745 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 11:55:48,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:48,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:48,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:48,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:48,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:48,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:48,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:48,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:48,747 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 11:55:48,747 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 11:55:48,747 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:48,747 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:48,747 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 11:55:48,747 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:48,749 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:48,749 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:48,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:48,749 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:48,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:48,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:48,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:48,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:48,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:48,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:48,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:48,750 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:48,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:48,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:48,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:48,751 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:48,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:48,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,752 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,753 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:48,753 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:48,753 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:48,753 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:48,753 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 11:55:48,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:48,753 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:48,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:48,753 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:48,753 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:48,753 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:48,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 11:55:48,753 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:48,753 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:48,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:48,853 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:48,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:48,865 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:48,874 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:48,883 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:48,884 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:48,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:55:48,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 11:55:48,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 11:55:48,911 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 11:55:48,911 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:48,911 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 11:55:48,911 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 11:55:48,912 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:48,912 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:48,913 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,913 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:48,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:48,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:48,913 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:48,913 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:55:48,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:48,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:48,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:55:48,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:48,914 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 11:55:48,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 11:55:48,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 11:55:48,914 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 11:55:48,914 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:55:48,914 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 11:55:48,915 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:48,915 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:48,915 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:48,915 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. started... 2023-03-08 11:55:48,915 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:55:48,915 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:55:48,915 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. started... 2023-03-08 11:55:48,915 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. started... 2023-03-08 11:55:48,915 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 64864abb7d0098b010ede90e39027ce9 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 11:55:48,915 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b617fde530c0c1bebf3fa2fa6eab77bb 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 11:55:48,915 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6249f9a64ffef0dbc4b217b148070179 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 11:55:49,334 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/e5428f178f27432889ceec0f067eb07e 2023-03-08 11:55:49,335 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/86aa12bafdba4727b5ad48b6e3ba6c9f 2023-03-08 11:55:49,338 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e5428f178f27432889ceec0f067eb07e 2023-03-08 11:55:49,338 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86aa12bafdba4727b5ad48b6e3ba6c9f 2023-03-08 11:55:49,731 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/5d0e13c2ccdc47b88bc66a24cd0220ab 2023-03-08 11:55:49,735 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d0e13c2ccdc47b88bc66a24cd0220ab 2023-03-08 11:55:50,154 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/c89a39de956849e697a031a82256f05f 2023-03-08 11:55:50,154 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/caf1a84ff2fb427dabd68a713afbb21b 2023-03-08 11:55:50,158 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for caf1a84ff2fb427dabd68a713afbb21b 2023-03-08 11:55:50,158 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c89a39de956849e697a031a82256f05f 2023-03-08 11:55:50,158 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/e5428f178f27432889ceec0f067eb07e as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/e5428f178f27432889ceec0f067eb07e 2023-03-08 11:55:50,158 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/86aa12bafdba4727b5ad48b6e3ba6c9f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/86aa12bafdba4727b5ad48b6e3ba6c9f 2023-03-08 11:55:50,162 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86aa12bafdba4727b5ad48b6e3ba6c9f 2023-03-08 11:55:50,162 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/86aa12bafdba4727b5ad48b6e3ba6c9f, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 11:55:50,162 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e5428f178f27432889ceec0f067eb07e 2023-03-08 11:55:50,162 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/e5428f178f27432889ceec0f067eb07e, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 11:55:50,163 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/c89a39de956849e697a031a82256f05f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/c89a39de956849e697a031a82256f05f 2023-03-08 11:55:50,163 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/caf1a84ff2fb427dabd68a713afbb21b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/caf1a84ff2fb427dabd68a713afbb21b 2023-03-08 11:55:50,166 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c89a39de956849e697a031a82256f05f 2023-03-08 11:55:50,166 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/c89a39de956849e697a031a82256f05f, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 11:55:50,166 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for caf1a84ff2fb427dabd68a713afbb21b 2023-03-08 11:55:50,166 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/caf1a84ff2fb427dabd68a713afbb21b, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 11:55:50,167 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 64864abb7d0098b010ede90e39027ce9 in 1252ms, sequenceid=23, compaction requested=true 2023-03-08 11:55:50,167 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for b617fde530c0c1bebf3fa2fa6eab77bb in 1252ms, sequenceid=23, compaction requested=true 2023-03-08 11:55:50,167 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:55:50,167 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:50,167 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:55:50,167 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:50,167 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:55:50,167 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:55:50,547 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/f75c2f2de243493f9d84f0b70652031b 2023-03-08 11:55:50,551 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f75c2f2de243493f9d84f0b70652031b 2023-03-08 11:55:50,551 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/5d0e13c2ccdc47b88bc66a24cd0220ab as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/5d0e13c2ccdc47b88bc66a24cd0220ab 2023-03-08 11:55:50,554 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d0e13c2ccdc47b88bc66a24cd0220ab 2023-03-08 11:55:50,554 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/5d0e13c2ccdc47b88bc66a24cd0220ab, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 11:55:50,555 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/f75c2f2de243493f9d84f0b70652031b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/f75c2f2de243493f9d84f0b70652031b 2023-03-08 11:55:50,558 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f75c2f2de243493f9d84f0b70652031b 2023-03-08 11:55:50,558 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/f75c2f2de243493f9d84f0b70652031b, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 11:55:50,558 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6249f9a64ffef0dbc4b217b148070179 in 1643ms, sequenceid=23, compaction requested=true 2023-03-08 11:55:50,558 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:55:50,558 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:50,558 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:55:50,558 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:55:50,559 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,559 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 11:55:50,559 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 11:55:50,560 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:50,560 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:50,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:50,560 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:50,560 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:55:50,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:50,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:50,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:50,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:50,561 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 11:55:50,561 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e43514b[Count = 0] remaining members to acquire global barrier 2023-03-08 11:55:50,561 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 11:55:50,561 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:50,563 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:50,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:50,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:50,563 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 11:55:50,563 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 11:55:50,563 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:55:50,563 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,563 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:55:50,565 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,565 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 11:55:50,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:50,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:50,565 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:50,565 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 11:55:50,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:50,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:50,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:50,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:50,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:50,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:55:50,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 11:55:50,567 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 11:55:50,567 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:55:50,567 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:55:50,567 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 11:55:50,567 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:55:50,571 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:50,571 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:50,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:50,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:50,571 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:50,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:50,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:55:50,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:55:50,571 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:50,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:50,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:55:50,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:50,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:50,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:55:50,572 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:50,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:55:50,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 11:55:50,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,574 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,574 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:50,574 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:55:50,574 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 11:55:50,574 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:55:50,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 11:55:50,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:55:50,574 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:55:50,574 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:50,574 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 11:55:50,574 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 11:55:50,574 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 11:55:50,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:55:50,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:55:50,574 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:55:50,574 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 11:55:50,575 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:55:50,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:55:50,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:55:50,675 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:55:50,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:55:50,676 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:50,676 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): 64864abb7d0098b010ede90e39027ce9/family1 is initiating major compaction (all files) 2023-03-08 11:55:50,676 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:50,676 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:50,676 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of 64864abb7d0098b010ede90e39027ce9/family1 in ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:50,677 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/be342fae247642e2b85432ca186c01b2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2bc6fe67a3e2487d856842de3f07f2e6, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/e5e3731fb27c43f2a7b7f89cf5a43e63, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2d3bbaff12904dea8f7bfc28b175675c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/86aa12bafdba4727b5ad48b6e3ba6c9f] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp, totalSize=26.6 K 2023-03-08 11:55:50,677 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting be342fae247642e2b85432ca186c01b2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276541681 2023-03-08 11:55:50,677 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 2bc6fe67a3e2487d856842de3f07f2e6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276543498 2023-03-08 11:55:50,678 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting e5e3731fb27c43f2a7b7f89cf5a43e63, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276545288 2023-03-08 11:55:50,678 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 2d3bbaff12904dea8f7bfc28b175675c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276547082 2023-03-08 11:55:50,678 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 86aa12bafdba4727b5ad48b6e3ba6c9f, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:55:50,688 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): 64864abb7d0098b010ede90e39027ce9#family1#compaction#148 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:51,095 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303085b32d0512c644372a1b939f0f315f75b_del 2023-03-08 11:55:51,095 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303085b32d0512c644372a1b939f0f315f75b_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303085b32d0512c644372a1b939f0f315f75b_del 2023-03-08 11:55:51,907 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 11d9119aff13424185235c8be98fefd9 2023-03-08 11:55:51,908 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family1/11d9119aff13424185235c8be98fefd9 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/11d9119aff13424185235c8be98fefd9 2023-03-08 11:55:51,911 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 11d9119aff13424185235c8be98fefd9 2023-03-08 11:55:51,912 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 64864abb7d0098b010ede90e39027ce9/family1 of 64864abb7d0098b010ede90e39027ce9 into 11d9119aff13424185235c8be98fefd9(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:51,912 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:55:51,912 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:51,912 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): 64864abb7d0098b010ede90e39027ce9/family2 is initiating major compaction (all files) 2023-03-08 11:55:51,912 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:51,912 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:51,913 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of 64864abb7d0098b010ede90e39027ce9/family2 in ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:55:51,913 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/25514da1a31145e98f50eaf5ee044b18, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/ee5dc82f18f84556a736f46877971086, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/643e438faed94d898daee210d4232d9d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/06914855ae444a38b3b4728b06307e8d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/c89a39de956849e697a031a82256f05f] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp, totalSize=25.3 K 2023-03-08 11:55:51,913 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 25514da1a31145e98f50eaf5ee044b18, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276541681 2023-03-08 11:55:51,913 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting ee5dc82f18f84556a736f46877971086, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276543498 2023-03-08 11:55:51,913 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 643e438faed94d898daee210d4232d9d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276545288 2023-03-08 11:55:51,914 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 06914855ae444a38b3b4728b06307e8d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276547082 2023-03-08 11:55:51,914 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting c89a39de956849e697a031a82256f05f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:55:51,921 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): 64864abb7d0098b010ede90e39027ce9#family2#compaction#149 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:51,974 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/be342fae247642e2b85432ca186c01b2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2bc6fe67a3e2487d856842de3f07f2e6, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/e5e3731fb27c43f2a7b7f89cf5a43e63, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2d3bbaff12904dea8f7bfc28b175675c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/86aa12bafdba4727b5ad48b6e3ba6c9f] to archive 2023-03-08 11:55:51,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:51,977 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/be342fae247642e2b85432ca186c01b2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/be342fae247642e2b85432ca186c01b2 2023-03-08 11:55:51,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2bc6fe67a3e2487d856842de3f07f2e6 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2bc6fe67a3e2487d856842de3f07f2e6 2023-03-08 11:55:51,979 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/e5e3731fb27c43f2a7b7f89cf5a43e63 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/e5e3731fb27c43f2a7b7f89cf5a43e63 2023-03-08 11:55:51,980 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2d3bbaff12904dea8f7bfc28b175675c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/2d3bbaff12904dea8f7bfc28b175675c 2023-03-08 11:55:51,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/86aa12bafdba4727b5ad48b6e3ba6c9f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/86aa12bafdba4727b5ad48b6e3ba6c9f 2023-03-08 11:55:52,329 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308b8c956915c074a11b5e99ec16f07b203_del 2023-03-08 11:55:52,329 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308b8c956915c074a11b5e99ec16f07b203_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230308b8c956915c074a11b5e99ec16f07b203_del 2023-03-08 11:55:53,141 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f5aabdcd362345ab94f7822c2ce248ef 2023-03-08 11:55:53,142 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/.tmp/family2/f5aabdcd362345ab94f7822c2ce248ef as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/f5aabdcd362345ab94f7822c2ce248ef 2023-03-08 11:55:53,145 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f5aabdcd362345ab94f7822c2ce248ef 2023-03-08 11:55:53,146 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 64864abb7d0098b010ede90e39027ce9/family2 of 64864abb7d0098b010ede90e39027ce9 into f5aabdcd362345ab94f7822c2ce248ef(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:53,146 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:55:53,146 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:53,146 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): b617fde530c0c1bebf3fa2fa6eab77bb/family1 is initiating major compaction (all files) 2023-03-08 11:55:53,146 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:53,146 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:53,147 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of b617fde530c0c1bebf3fa2fa6eab77bb/family1 in ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:53,147 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/2cd44157d98849529cfbf008d6bfd7ef, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/afc4ddbf11684bcab57844e45b7bc7fe, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/dca727f6da6b4ada8158f6d219ffd2d6, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/beea862a0d664985af3914d0e74f5eb2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/e5428f178f27432889ceec0f067eb07e] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp, totalSize=26.6 K 2023-03-08 11:55:53,147 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 2cd44157d98849529cfbf008d6bfd7ef, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276541681 2023-03-08 11:55:53,148 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting afc4ddbf11684bcab57844e45b7bc7fe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276543498 2023-03-08 11:55:53,148 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting dca727f6da6b4ada8158f6d219ffd2d6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276545288 2023-03-08 11:55:53,148 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting beea862a0d664985af3914d0e74f5eb2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276547082 2023-03-08 11:55:53,149 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting e5428f178f27432889ceec0f067eb07e, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:55:53,156 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): b617fde530c0c1bebf3fa2fa6eab77bb#family1#compaction#150 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:53,172 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/25514da1a31145e98f50eaf5ee044b18, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/ee5dc82f18f84556a736f46877971086, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/643e438faed94d898daee210d4232d9d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/06914855ae444a38b3b4728b06307e8d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/c89a39de956849e697a031a82256f05f] to archive 2023-03-08 11:55:53,173 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:53,174 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/25514da1a31145e98f50eaf5ee044b18 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/25514da1a31145e98f50eaf5ee044b18 2023-03-08 11:55:53,175 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/ee5dc82f18f84556a736f46877971086 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/ee5dc82f18f84556a736f46877971086 2023-03-08 11:55:53,176 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/643e438faed94d898daee210d4232d9d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/643e438faed94d898daee210d4232d9d 2023-03-08 11:55:53,177 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/06914855ae444a38b3b4728b06307e8d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/06914855ae444a38b3b4728b06307e8d 2023-03-08 11:55:53,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/c89a39de956849e697a031a82256f05f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family2/c89a39de956849e697a031a82256f05f 2023-03-08 11:55:53,564 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030849daf0fa0f284c7688fd22b7a8f3f8d0_del 2023-03-08 11:55:53,564 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030849daf0fa0f284c7688fd22b7a8f3f8d0_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023030849daf0fa0f284c7688fd22b7a8f3f8d0_del 2023-03-08 11:55:54,377 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97804dcd9a484b9c9ab2145e2d48ad10 2023-03-08 11:55:54,378 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family1/97804dcd9a484b9c9ab2145e2d48ad10 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/97804dcd9a484b9c9ab2145e2d48ad10 2023-03-08 11:55:54,381 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97804dcd9a484b9c9ab2145e2d48ad10 2023-03-08 11:55:54,382 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b617fde530c0c1bebf3fa2fa6eab77bb/family1 of b617fde530c0c1bebf3fa2fa6eab77bb into 97804dcd9a484b9c9ab2145e2d48ad10(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:54,382 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:55:54,382 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:54,382 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): b617fde530c0c1bebf3fa2fa6eab77bb/family2 is initiating major compaction (all files) 2023-03-08 11:55:54,382 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:54,382 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:54,383 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of b617fde530c0c1bebf3fa2fa6eab77bb/family2 in ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:55:54,383 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/efb5dae7e2f746a8a5a56dd52fc4a4dc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/da7bdf919dc84f03a829a85bc7d26f91, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/340d9effbdfa40d2b5e2f84752eb03ec, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/109e80bc9feb4bf283653bc2b6a2e5a7, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/caf1a84ff2fb427dabd68a713afbb21b] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp, totalSize=25.3 K 2023-03-08 11:55:54,383 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting efb5dae7e2f746a8a5a56dd52fc4a4dc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276541681 2023-03-08 11:55:54,383 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting da7bdf919dc84f03a829a85bc7d26f91, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276543498 2023-03-08 11:55:54,384 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 340d9effbdfa40d2b5e2f84752eb03ec, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276545288 2023-03-08 11:55:54,384 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 109e80bc9feb4bf283653bc2b6a2e5a7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276547082 2023-03-08 11:55:54,384 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting caf1a84ff2fb427dabd68a713afbb21b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:55:54,392 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): b617fde530c0c1bebf3fa2fa6eab77bb#family2#compaction#151 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:54,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/2cd44157d98849529cfbf008d6bfd7ef, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/afc4ddbf11684bcab57844e45b7bc7fe, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/dca727f6da6b4ada8158f6d219ffd2d6, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/beea862a0d664985af3914d0e74f5eb2, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/e5428f178f27432889ceec0f067eb07e] to archive 2023-03-08 11:55:54,481 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:54,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/2cd44157d98849529cfbf008d6bfd7ef to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/2cd44157d98849529cfbf008d6bfd7ef 2023-03-08 11:55:54,483 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/afc4ddbf11684bcab57844e45b7bc7fe to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/afc4ddbf11684bcab57844e45b7bc7fe 2023-03-08 11:55:54,484 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/dca727f6da6b4ada8158f6d219ffd2d6 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/dca727f6da6b4ada8158f6d219ffd2d6 2023-03-08 11:55:54,485 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/beea862a0d664985af3914d0e74f5eb2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/beea862a0d664985af3914d0e74f5eb2 2023-03-08 11:55:54,486 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/e5428f178f27432889ceec0f067eb07e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/e5428f178f27432889ceec0f067eb07e 2023-03-08 11:55:54,800 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303088dc3cda6fecb4973914e6733f75890c7_del 2023-03-08 11:55:54,800 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303088dc3cda6fecb4973914e6733f75890c7_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303088dc3cda6fecb4973914e6733f75890c7_del 2023-03-08 11:55:55,614 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3c2a63ea042542c480c64d1e02fc7756 2023-03-08 11:55:55,615 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/.tmp/family2/3c2a63ea042542c480c64d1e02fc7756 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/3c2a63ea042542c480c64d1e02fc7756 2023-03-08 11:55:55,618 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3c2a63ea042542c480c64d1e02fc7756 2023-03-08 11:55:55,620 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b617fde530c0c1bebf3fa2fa6eab77bb/family2 of b617fde530c0c1bebf3fa2fa6eab77bb into 3c2a63ea042542c480c64d1e02fc7756(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:55,620 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:55:55,620 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:55,620 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): 6249f9a64ffef0dbc4b217b148070179/family1 is initiating major compaction (all files) 2023-03-08 11:55:55,620 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:55,620 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:55,620 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of 6249f9a64ffef0dbc4b217b148070179/family1 in ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:55,620 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/3f38423a484a464591de987cabbc5e59, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/2c11019d72a6400a853fed707ae43e83, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/98785f26ff904586a17880243dcbfd5f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/94c459887801443aafb04ca3498ac6d1, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/5d0e13c2ccdc47b88bc66a24cd0220ab] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp, totalSize=26.6 K 2023-03-08 11:55:55,621 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 3f38423a484a464591de987cabbc5e59, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276541681 2023-03-08 11:55:55,621 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 2c11019d72a6400a853fed707ae43e83, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276543498 2023-03-08 11:55:55,621 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 98785f26ff904586a17880243dcbfd5f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276545288 2023-03-08 11:55:55,622 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 94c459887801443aafb04ca3498ac6d1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276547082 2023-03-08 11:55:55,622 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 5d0e13c2ccdc47b88bc66a24cd0220ab, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:55:55,631 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): 6249f9a64ffef0dbc4b217b148070179#family1#compaction#152 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:55,672 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/efb5dae7e2f746a8a5a56dd52fc4a4dc, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/da7bdf919dc84f03a829a85bc7d26f91, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/340d9effbdfa40d2b5e2f84752eb03ec, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/109e80bc9feb4bf283653bc2b6a2e5a7, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/caf1a84ff2fb427dabd68a713afbb21b] to archive 2023-03-08 11:55:55,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:55,675 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/efb5dae7e2f746a8a5a56dd52fc4a4dc to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/efb5dae7e2f746a8a5a56dd52fc4a4dc 2023-03-08 11:55:55,676 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/da7bdf919dc84f03a829a85bc7d26f91 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/da7bdf919dc84f03a829a85bc7d26f91 2023-03-08 11:55:55,677 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/340d9effbdfa40d2b5e2f84752eb03ec to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/340d9effbdfa40d2b5e2f84752eb03ec 2023-03-08 11:55:55,678 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/109e80bc9feb4bf283653bc2b6a2e5a7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/109e80bc9feb4bf283653bc2b6a2e5a7 2023-03-08 11:55:55,679 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/caf1a84ff2fb427dabd68a713afbb21b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family2/caf1a84ff2fb427dabd68a713afbb21b 2023-03-08 11:55:56,038 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023030870f3d67ae45448fb919de0dd8c4b865d_del 2023-03-08 11:55:56,039 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030870f3d67ae45448fb919de0dd8c4b865d_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023030870f3d67ae45448fb919de0dd8c4b865d_del 2023-03-08 11:55:56,852 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60b99abd410b46b3aedb80c2ba310e61 2023-03-08 11:55:56,853 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family1/60b99abd410b46b3aedb80c2ba310e61 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/60b99abd410b46b3aedb80c2ba310e61 2023-03-08 11:55:56,856 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60b99abd410b46b3aedb80c2ba310e61 2023-03-08 11:55:56,857 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6249f9a64ffef0dbc4b217b148070179/family1 of 6249f9a64ffef0dbc4b217b148070179 into 60b99abd410b46b3aedb80c2ba310e61(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:56,857 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:55:56,857 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:55:56,857 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): 6249f9a64ffef0dbc4b217b148070179/family2 is initiating major compaction (all files) 2023-03-08 11:55:56,857 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:55:56,857 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:55:56,858 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of 6249f9a64ffef0dbc4b217b148070179/family2 in ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:55:56,858 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/d5cc3f3ddcdf4c14ab934a7b682577ea, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/51c55a53b9c94221bf1b0d5f5932aae5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/81effd51b5354aaca70d4b9e1f5ff66d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/567b9d336aff4f1885bdbd8c4a6ef933, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/f75c2f2de243493f9d84f0b70652031b] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp, totalSize=25.3 K 2023-03-08 11:55:56,858 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting d5cc3f3ddcdf4c14ab934a7b682577ea, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276541681 2023-03-08 11:55:56,858 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 51c55a53b9c94221bf1b0d5f5932aae5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276543498 2023-03-08 11:55:56,858 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 81effd51b5354aaca70d4b9e1f5ff66d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276545288 2023-03-08 11:55:56,859 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 567b9d336aff4f1885bdbd8c4a6ef933, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276547082 2023-03-08 11:55:56,859 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting f75c2f2de243493f9d84f0b70652031b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:55:56,866 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): 6249f9a64ffef0dbc4b217b148070179#family2#compaction#153 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:55:56,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/3f38423a484a464591de987cabbc5e59, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/2c11019d72a6400a853fed707ae43e83, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/98785f26ff904586a17880243dcbfd5f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/94c459887801443aafb04ca3498ac6d1, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/5d0e13c2ccdc47b88bc66a24cd0220ab] to archive 2023-03-08 11:55:56,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:56,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/3f38423a484a464591de987cabbc5e59 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/3f38423a484a464591de987cabbc5e59 2023-03-08 11:55:56,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/2c11019d72a6400a853fed707ae43e83 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/2c11019d72a6400a853fed707ae43e83 2023-03-08 11:55:56,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/98785f26ff904586a17880243dcbfd5f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/98785f26ff904586a17880243dcbfd5f 2023-03-08 11:55:56,878 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/94c459887801443aafb04ca3498ac6d1 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/94c459887801443aafb04ca3498ac6d1 2023-03-08 11:55:56,879 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/5d0e13c2ccdc47b88bc66a24cd0220ab to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/5d0e13c2ccdc47b88bc66a24cd0220ab 2023-03-08 11:55:57,274 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023030890adf9da8e634746be7b9ca386edfdc4_del 2023-03-08 11:55:57,274 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030890adf9da8e634746be7b9ca386edfdc4_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023030890adf9da8e634746be7b9ca386edfdc4_del 2023-03-08 11:55:58,088 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09441e2cdcbc42c1b51d68dde81b81db 2023-03-08 11:55:58,089 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/.tmp/family2/09441e2cdcbc42c1b51d68dde81b81db as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/09441e2cdcbc42c1b51d68dde81b81db 2023-03-08 11:55:58,093 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09441e2cdcbc42c1b51d68dde81b81db 2023-03-08 11:55:58,093 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6249f9a64ffef0dbc4b217b148070179/family2 of 6249f9a64ffef0dbc4b217b148070179 into 09441e2cdcbc42c1b51d68dde81b81db(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:55:58,094 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:55:58,097 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:58,100 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:58,104 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:58,105 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:58,109 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 11:55:58,121 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-08 11:55:58,123 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023030870f3d67ae45448fb919de0dd8c4b865d_del 2023-03-08 11:55:58,126 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030849daf0fa0f284c7688fd22b7a8f3f8d0_del 2023-03-08 11:55:58,129 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303085b32d0512c644372a1b939f0f315f75b_del 2023-03-08 11:55:58,129 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-08 11:55:58,130 DEBUG [Listener at localhost/36211] zookeeper.ReadOnlyZKClient(139): Connect 0x02465de9 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:55:58,134 DEBUG [Listener at localhost/36211] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@68e83979, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:55:58,136 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 11:55:58,136 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 11:55:58,136 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 11:55:58,167 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303087205238c9aed4a8dbda875b328ee6878 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303087205238c9aed4a8dbda875b328ee6878 2023-03-08 11:55:58,168 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 11:55:58,172 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/d5cc3f3ddcdf4c14ab934a7b682577ea, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/51c55a53b9c94221bf1b0d5f5932aae5, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/81effd51b5354aaca70d4b9e1f5ff66d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/567b9d336aff4f1885bdbd8c4a6ef933, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/f75c2f2de243493f9d84f0b70652031b] to archive 2023-03-08 11:55:58,173 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:58,174 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/d5cc3f3ddcdf4c14ab934a7b682577ea to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/d5cc3f3ddcdf4c14ab934a7b682577ea 2023-03-08 11:55:58,175 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/51c55a53b9c94221bf1b0d5f5932aae5 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/51c55a53b9c94221bf1b0d5f5932aae5 2023-03-08 11:55:58,176 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/81effd51b5354aaca70d4b9e1f5ff66d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/81effd51b5354aaca70d4b9e1f5ff66d 2023-03-08 11:55:58,177 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/567b9d336aff4f1885bdbd8c4a6ef933 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/567b9d336aff4f1885bdbd8c4a6ef933 2023-03-08 11:55:58,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/f75c2f2de243493f9d84f0b70652031b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family2/f75c2f2de243493f9d84f0b70652031b 2023-03-08 11:55:58,189 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:55:58,191 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51850, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:55:58,202 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:55:58,205 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53730, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:55:58,219 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/bf46b006d799467e9d4e0fdda14a6dff first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-08 11:55:58,220 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/bf46b006d799467e9d4e0fdda14a6dff}] 2023-03-08 11:55:58,227 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/bf46b006d799467e9d4e0fdda14a6dff for inclusion in 64864abb7d0098b010ede90e39027ce9/family1 2023-03-08 11:55:58,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-08 11:55:58,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 11:55:58,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:55:58,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/bf46b006d799467e9d4e0fdda14a6dff to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__efti36la3u3ouq7q6747723ac6kj271lplod2g75ej66gsj3cbvc05f016u1dei5/family1/bf46b006d799467e9d4e0fdda14a6dff 2023-03-08 11:55:58,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__efti36la3u3ouq7q6747723ac6kj271lplod2g75ej66gsj3cbvc05f016u1dei5/family1/bf46b006d799467e9d4e0fdda14a6dff as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/69e1268448ec4981b35e7b4139dbb946_SeqId_28_ 2023-03-08 11:55:58,235 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__efti36la3u3ouq7q6747723ac6kj271lplod2g75ej66gsj3cbvc05f016u1dei5/family1/bf46b006d799467e9d4e0fdda14a6dff into 64864abb7d0098b010ede90e39027ce9/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/69e1268448ec4981b35e7b4139dbb946_SeqId_28_ - updating store file list. 2023-03-08 11:55:58,238 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/69e1268448ec4981b35e7b4139dbb946_SeqId_28_ into 64864abb7d0098b010ede90e39027ce9/family1 2023-03-08 11:55:58,238 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__efti36la3u3ouq7q6747723ac6kj271lplod2g75ej66gsj3cbvc05f016u1dei5/family1/bf46b006d799467e9d4e0fdda14a6dff into 64864abb7d0098b010ede90e39027ce9/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/family1/69e1268448ec4981b35e7b4139dbb946_SeqId_28_) 2023-03-08 11:55:58,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__efti36la3u3ouq7q6747723ac6kj271lplod2g75ej66gsj3cbvc05f016u1dei5/family1/bf46b006d799467e9d4e0fdda14a6dff 2023-03-08 11:55:58,241 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 11:55:58,241 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:58,242 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308ccbc1fe6515e49c9a890703301b43830 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308ccbc1fe6515e49c9a890703301b43830 2023-03-08 11:55:58,243 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 11:55:58,243 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 11:55:58,243 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 11:55:58,968 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308dd4e753aa95c4bda8582138783538016 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308dd4e753aa95c4bda8582138783538016 2023-03-08 11:55:58,969 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 11:55:58,970 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303086d8f59da40494c66a24f3b69abe14eae to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303086d8f59da40494c66a24f3b69abe14eae 2023-03-08 11:55:58,971 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 11:55:59,002 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/4e05d2deb95644c0ade93b7d30ec0c7f first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-08 11:55:59,003 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/cc2d49f7be934f0ab1f777943e1349b8 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-08 11:55:59,003 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/4e05d2deb95644c0ade93b7d30ec0c7f}] 2023-03-08 11:55:59,003 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/cc2d49f7be934f0ab1f777943e1349b8}] 2023-03-08 11:55:59,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/cc2d49f7be934f0ab1f777943e1349b8 for inclusion in b617fde530c0c1bebf3fa2fa6eab77bb/family1 2023-03-08 11:55:59,009 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/4e05d2deb95644c0ade93b7d30ec0c7f for inclusion in 6249f9a64ffef0dbc4b217b148070179/family1 2023-03-08 11:55:59,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-08 11:55:59,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 11:55:59,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-08 11:55:59,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 11:55:59,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:55:59,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:55:59,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/4e05d2deb95644c0ade93b7d30ec0c7f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__5n7a91is9mdsofliei88ibpdi9rkh9gtg4dklatetfaqu9nthu963dpmor9ldiu3/family1/4e05d2deb95644c0ade93b7d30ec0c7f 2023-03-08 11:55:59,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/cc2d49f7be934f0ab1f777943e1349b8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__8qjg4ldqbvrll85dp6lcs6niqrv6vnku808t8af4jbeihecqv9tf56vboc87uidp/family1/cc2d49f7be934f0ab1f777943e1349b8 2023-03-08 11:55:59,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__5n7a91is9mdsofliei88ibpdi9rkh9gtg4dklatetfaqu9nthu963dpmor9ldiu3/family1/4e05d2deb95644c0ade93b7d30ec0c7f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/62b2e3a3736948788fa90a7d6c536db4_SeqId_28_ 2023-03-08 11:55:59,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__8qjg4ldqbvrll85dp6lcs6niqrv6vnku808t8af4jbeihecqv9tf56vboc87uidp/family1/cc2d49f7be934f0ab1f777943e1349b8 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/4a2553f119cd4dbb92eab99e6329c5df_SeqId_28_ 2023-03-08 11:55:59,016 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__5n7a91is9mdsofliei88ibpdi9rkh9gtg4dklatetfaqu9nthu963dpmor9ldiu3/family1/4e05d2deb95644c0ade93b7d30ec0c7f into 6249f9a64ffef0dbc4b217b148070179/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/62b2e3a3736948788fa90a7d6c536db4_SeqId_28_ - updating store file list. 2023-03-08 11:55:59,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__8qjg4ldqbvrll85dp6lcs6niqrv6vnku808t8af4jbeihecqv9tf56vboc87uidp/family1/cc2d49f7be934f0ab1f777943e1349b8 into b617fde530c0c1bebf3fa2fa6eab77bb/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/4a2553f119cd4dbb92eab99e6329c5df_SeqId_28_ - updating store file list. 2023-03-08 11:55:59,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/4a2553f119cd4dbb92eab99e6329c5df_SeqId_28_ into b617fde530c0c1bebf3fa2fa6eab77bb/family1 2023-03-08 11:55:59,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__8qjg4ldqbvrll85dp6lcs6niqrv6vnku808t8af4jbeihecqv9tf56vboc87uidp/family1/cc2d49f7be934f0ab1f777943e1349b8 into b617fde530c0c1bebf3fa2fa6eab77bb/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/family1/4a2553f119cd4dbb92eab99e6329c5df_SeqId_28_) 2023-03-08 11:55:59,020 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/62b2e3a3736948788fa90a7d6c536db4_SeqId_28_ into 6249f9a64ffef0dbc4b217b148070179/family1 2023-03-08 11:55:59,020 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__5n7a91is9mdsofliei88ibpdi9rkh9gtg4dklatetfaqu9nthu963dpmor9ldiu3/family1/4e05d2deb95644c0ade93b7d30ec0c7f into 6249f9a64ffef0dbc4b217b148070179/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/family1/62b2e3a3736948788fa90a7d6c536db4_SeqId_28_) 2023-03-08 11:55:59,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__8qjg4ldqbvrll85dp6lcs6niqrv6vnku808t8af4jbeihecqv9tf56vboc87uidp/family1/cc2d49f7be934f0ab1f777943e1349b8 2023-03-08 11:55:59,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__ns_testMinorCompaction__5n7a91is9mdsofliei88ibpdi9rkh9gtg4dklatetfaqu9nthu963dpmor9ldiu3/family1/4e05d2deb95644c0ade93b7d30ec0c7f 2023-03-08 11:55:59,022 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 11:55:59,022 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 11:55:59,023 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:59,023 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:55:59,024 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308c171e2359512498297f58232adc70e02 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308c171e2359512498297f58232adc70e02 2023-03-08 11:55:59,024 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 11:55:59,024 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 11:55:59,024 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308ec0edea7c87442c1ab175026ad017076 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308ec0edea7c87442c1ab175026ad017076 2023-03-08 11:55:59,025 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 11:55:59,025 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 11:55:59,025 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 11:55:59,025 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 11:55:59,025 INFO [Listener at localhost/36211] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-08 11:55:59,027 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:59,030 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:59,032 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:55:59,033 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:55:59,050 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=254 (was 285), OpenFileDescriptor=638 (was 769), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=150 (was 171), ProcessCount=173 (was 170) - ProcessCount LEAK? -, AvailableMemoryMB=7366 (was 7402) 2023-03-08 11:55:59,061 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=254, OpenFileDescriptor=638, MaxFileDescriptor=60000, SystemLoadAverage=150, ProcessCount=173, AvailableMemoryMB=7365 2023-03-08 11:55:59,063 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 11:55:59,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-08 11:55:59,065 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:55:59,066 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-08 11:55:59,066 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:55:59,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 11:55:59,067 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192 2023-03-08 11:55:59,068 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192 empty. 2023-03-08 11:55:59,068 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192 2023-03-08 11:55:59,068 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-08 11:55:59,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 11:55:59,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 11:55:59,475 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 11:55:59,477 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => e53083d9a161a7082c38a975fc8a0192, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:55:59,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 11:55:59,882 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:55:59,882 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing e53083d9a161a7082c38a975fc8a0192, disabling compactions & flushes 2023-03-08 11:55:59,882 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:55:59,882 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:55:59,882 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. after waiting 0 ms 2023-03-08 11:55:59,882 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:55:59,882 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:55:59,882 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:55:59,884 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:55:59,885 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678276559885"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276559885"}]},"ts":"1678276559885"} 2023-03-08 11:55:59,887 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 11:55:59,887 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:55:59,888 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276559887"}]},"ts":"1678276559887"} 2023-03-08 11:55:59,889 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-08 11:55:59,893 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=e53083d9a161a7082c38a975fc8a0192, ASSIGN}] 2023-03-08 11:55:59,894 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=e53083d9a161a7082c38a975fc8a0192, ASSIGN 2023-03-08 11:55:59,895 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=e53083d9a161a7082c38a975fc8a0192, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:56:00,046 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=e53083d9a161a7082c38a975fc8a0192, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:00,046 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678276560046"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276560046"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276560046"}]},"ts":"1678276560046"} 2023-03-08 11:56:00,048 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure e53083d9a161a7082c38a975fc8a0192, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:56:00,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 11:56:00,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:00,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e53083d9a161a7082c38a975fc8a0192, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192.', STARTKEY => '', ENDKEY => ''} 2023-03-08 11:56:00,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy e53083d9a161a7082c38a975fc8a0192 2023-03-08 11:56:00,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:56:00,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e53083d9a161a7082c38a975fc8a0192 2023-03-08 11:56:00,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e53083d9a161a7082c38a975fc8a0192 2023-03-08 11:56:00,203 INFO [StoreOpener-e53083d9a161a7082c38a975fc8a0192-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e53083d9a161a7082c38a975fc8a0192 2023-03-08 11:56:00,204 DEBUG [StoreOpener-e53083d9a161a7082c38a975fc8a0192-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1 2023-03-08 11:56:00,204 DEBUG [StoreOpener-e53083d9a161a7082c38a975fc8a0192-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1 2023-03-08 11:56:00,204 INFO [StoreOpener-e53083d9a161a7082c38a975fc8a0192-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e53083d9a161a7082c38a975fc8a0192 columnFamilyName family1 2023-03-08 11:56:00,206 INFO [StoreOpener-e53083d9a161a7082c38a975fc8a0192-1] regionserver.HStore(310): Store=e53083d9a161a7082c38a975fc8a0192/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:56:00,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192 2023-03-08 11:56:00,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192 2023-03-08 11:56:00,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e53083d9a161a7082c38a975fc8a0192 2023-03-08 11:56:00,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:56:00,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e53083d9a161a7082c38a975fc8a0192; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63377162, jitterRate=-0.055606693029403687}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 11:56:00,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:00,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192., pid=114, masterSystemTime=1678276560199 2023-03-08 11:56:00,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:00,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:00,212 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=e53083d9a161a7082c38a975fc8a0192, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:00,212 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678276560212"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276560212"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276560212"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276560212"}]},"ts":"1678276560212"} 2023-03-08 11:56:00,214 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-08 11:56:00,215 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure e53083d9a161a7082c38a975fc8a0192, server=jenkins-hbase4.apache.org,45551,1678276414946 in 165 msec 2023-03-08 11:56:00,216 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-08 11:56:00,216 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=e53083d9a161a7082c38a975fc8a0192, ASSIGN in 322 msec 2023-03-08 11:56:00,216 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:56:00,216 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276560216"}]},"ts":"1678276560216"} 2023-03-08 11:56:00,217 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-08 11:56:00,219 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:56:00,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1560 sec 2023-03-08 11:56:00,547 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:01,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 11:56:01,172 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-08 11:56:01,190 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:01,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:01,192 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:01,192 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:01,192 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:56:01,192 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:56:01,192 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:01,192 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:01,193 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:01,193 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:01,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:01,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:01,194 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:01,194 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:01,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:01,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:01,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:01,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:01,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:01,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:01,195 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:56:01,195 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:01,195 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:56:01,196 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:56:01,196 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:01,196 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:01,196 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. started... 2023-03-08 11:56:01,197 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e53083d9a161a7082c38a975fc8a0192 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 11:56:01,621 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511301d75aef396fa4c409ba432824a9cc4bd to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511301d75aef396fa4c409ba432824a9cc4bd 2023-03-08 11:56:02,024 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/dc3f445526154aff9c80a3408277dd71 2023-03-08 11:56:02,028 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/dc3f445526154aff9c80a3408277dd71 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/dc3f445526154aff9c80a3408277dd71 2023-03-08 11:56:02,032 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/dc3f445526154aff9c80a3408277dd71, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-08 11:56:02,033 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for e53083d9a161a7082c38a975fc8a0192 in 836ms, sequenceid=5, compaction requested=false 2023-03-08 11:56:02,033 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:02,034 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:02,034 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:56:02,034 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:56:02,034 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,034 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:56:02,034 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:56:02,035 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,035 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:02,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:02,035 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,035 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:02,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:02,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:56:02,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bbe7b0a[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:02,037 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:56:02,037 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,038 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,038 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:56:02,038 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:56:02,038 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:02,038 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,038 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:02,039 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,039 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:56:02,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:02,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:02,039 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:02,039 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:56:02,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:02,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:02,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:02,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:02,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:56:02,041 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:56:02,041 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:02,041 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:02,041 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,041 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:02,044 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,044 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:02,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:02,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,044 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:02,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:02,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:02,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:02,044 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:02,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,045 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:02,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,047 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,047 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,047 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,047 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,047 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:02,047 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,047 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:02,047 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:02,047 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:02,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:02,047 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:02,047 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:02,048 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,048 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:02,048 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:02,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:02,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:02,148 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:02,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:02,153 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:02,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:02,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,154 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:02,155 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:02,155 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:56:02,155 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:56:02,155 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,155 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,156 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:02,156 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:02,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:02,156 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,156 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:02,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:02,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,157 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,157 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:56:02,158 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,158 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:56:02,158 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:02,158 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:56:02,158 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:56:02,158 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:02,158 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:02,158 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. started... 2023-03-08 11:56:02,158 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e53083d9a161a7082c38a975fc8a0192 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 11:56:02,568 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112814748d6d963f4ae489020cf9769f4296 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112814748d6d963f4ae489020cf9769f4296 2023-03-08 11:56:02,973 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/8987500102bb438390f1a6b6033873a6 2023-03-08 11:56:02,977 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/8987500102bb438390f1a6b6033873a6 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/8987500102bb438390f1a6b6033873a6 2023-03-08 11:56:02,980 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/8987500102bb438390f1a6b6033873a6, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 11:56:02,981 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for e53083d9a161a7082c38a975fc8a0192 in 823ms, sequenceid=9, compaction requested=false 2023-03-08 11:56:02,981 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:02,981 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:02,981 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:56:02,981 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:56:02,981 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,981 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:56:02,981 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:56:02,982 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,982 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:02,983 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,983 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:02,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:56:02,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74426435[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:02,984 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:56:02,984 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,991 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,991 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:56:02,991 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:56:02,991 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:02,991 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,991 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:02,992 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:56:02,992 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:02,992 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,992 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:56:02,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:02,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:02,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:02,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:56:02,994 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:56:02,994 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:02,994 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:02,994 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,994 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:02,996 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,996 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:02,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:02,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,996 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:02,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:02,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:02,996 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:02,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:02,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,997 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:02,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:02,999 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:02,999 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:02,999 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:02,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:02,999 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:56:02,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:02,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:02,999 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:02,999 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,000 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:03,000 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:02,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:03,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:03,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:03,100 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:03,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:03,104 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:03,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:03,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,106 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:03,106 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:03,106 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:56:03,106 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:56:03,106 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,106 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,112 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,112 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:03,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:03,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:03,112 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,112 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:03,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:03,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,113 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,113 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:56:03,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,113 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:56:03,113 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:03,113 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:56:03,113 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:56:03,113 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:03,114 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:03,114 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. started... 2023-03-08 11:56:03,114 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e53083d9a161a7082c38a975fc8a0192 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:56:03,524 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201ec69519ca7ee4da9a4073de5ed7c89ba to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201ec69519ca7ee4da9a4073de5ed7c89ba 2023-03-08 11:56:03,929 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/94bc870370844b52b7fb97e6698974be 2023-03-08 11:56:03,933 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/94bc870370844b52b7fb97e6698974be as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/94bc870370844b52b7fb97e6698974be 2023-03-08 11:56:03,936 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/94bc870370844b52b7fb97e6698974be, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 11:56:03,937 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e53083d9a161a7082c38a975fc8a0192 in 823ms, sequenceid=13, compaction requested=true 2023-03-08 11:56:03,937 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:03,937 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:56:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:56:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:56:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:56:03,939 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,939 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:03,939 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:03,939 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:03,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:56:03,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d71ae4f[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:03,940 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:56:03,940 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,941 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,941 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:56:03,941 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:56:03,941 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,941 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:03,941 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:03,944 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,944 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:56:03,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:03,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:03,944 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:03,944 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:56:03,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:03,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:03,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:03,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:03,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:56:03,946 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:56:03,946 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:03,946 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:03,946 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,946 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:03,947 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,947 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:03,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:03,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,947 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:03,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:03,948 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:03,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:03,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:03,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,949 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:03,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,950 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,950 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:03,950 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:03,951 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:03,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:56:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:03,951 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:03,951 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:03,951 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:03,951 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,951 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:03,951 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,951 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:03,951 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:03,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:03,951 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:03,951 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:04,051 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:04,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:04,055 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:04,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:04,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,057 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:04,057 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:04,057 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:56:04,057 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:56:04,057 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,057 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,059 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:04,059 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:04,059 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,059 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,060 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:56:04,060 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,060 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:56:04,060 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:04,060 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:56:04,060 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:56:04,060 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:04,060 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:04,060 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. started... 2023-03-08 11:56:04,060 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e53083d9a161a7082c38a975fc8a0192 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:56:04,070 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054ceb5e24eb414013abb954cc8f3631e8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512054ceb5e24eb414013abb954cc8f3631e8 2023-03-08 11:56:04,084 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:56:04,085 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x02465de9 to 127.0.0.1:51863 2023-03-08 11:56:04,085 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:56:04,085 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:56:04,085 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3865866b to 127.0.0.1:51863 2023-03-08 11:56:04,085 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:56:04,086 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:56:04,086 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7c526350 to 127.0.0.1:51863 2023-03-08 11:56:04,086 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:56:04,489 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/6f843cbbd2364d6a97ceae59eeb04322 2023-03-08 11:56:04,493 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/6f843cbbd2364d6a97ceae59eeb04322 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6f843cbbd2364d6a97ceae59eeb04322 2023-03-08 11:56:04,496 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6f843cbbd2364d6a97ceae59eeb04322, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 11:56:04,496 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e53083d9a161a7082c38a975fc8a0192 in 436ms, sequenceid=17, compaction requested=true 2023-03-08 11:56:04,496 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:04,497 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:04,497 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:56:04,497 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:56:04,497 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,497 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:56:04,497 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:56:04,498 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,498 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:04,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:04,498 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,498 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:04,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:04,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:04,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:04,500 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:56:04,500 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39ac6ba3[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:04,500 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:56:04,500 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,501 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,501 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:56:04,501 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:56:04,501 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,501 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:04,501 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:04,503 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,503 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:56:04,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:04,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:04,503 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:04,503 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:56:04,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:04,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:04,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:04,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:04,505 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:56:04,505 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:56:04,505 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:04,505 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:04,505 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,505 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:04,507 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,507 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:04,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:04,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,507 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:04,507 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:04,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:04,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:04,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:04,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,509 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,511 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,511 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:04,511 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,511 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:04,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:04,511 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:04,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:04,511 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:04,511 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:04,511 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,511 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,511 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,511 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:04,511 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:04,512 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:04,512 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:04,612 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:04,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:04,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:04,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:04,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,617 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:04,617 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:04,617 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:56:04,617 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:56:04,618 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,618 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,619 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,619 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:04,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:04,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:04,619 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:04,619 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:04,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:04,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,620 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:56:04,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:04,621 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:56:04,621 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:04,621 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:56:04,621 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:56:04,621 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:04,621 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:04,621 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. started... 2023-03-08 11:56:04,621 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e53083d9a161a7082c38a975fc8a0192 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:56:05,032 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512286141dc9d2d764eab8b59576ad297143a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512286141dc9d2d764eab8b59576ad297143a 2023-03-08 11:56:05,437 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/da54dfc5e5d84809bb803e654a48cc46 2023-03-08 11:56:05,441 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/da54dfc5e5d84809bb803e654a48cc46 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/da54dfc5e5d84809bb803e654a48cc46 2023-03-08 11:56:05,443 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/da54dfc5e5d84809bb803e654a48cc46, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 11:56:05,444 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e53083d9a161a7082c38a975fc8a0192 in 823ms, sequenceid=21, compaction requested=true 2023-03-08 11:56:05,444 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:05,444 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:56:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:56:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:56:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:56:05,446 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,446 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:05,447 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,447 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:05,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:56:05,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@611221b7[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:05,448 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:56:05,448 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,449 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,449 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:56:05,449 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:56:05,449 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:05,449 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,449 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:05,450 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:56:05,450 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,450 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:05,450 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:56:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:05,452 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:56:05,452 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:56:05,452 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:05,452 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:05,452 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,452 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:05,453 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,453 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,453 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:05,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:05,454 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,455 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:05,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,466 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:05,466 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:05,466 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:05,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:05,466 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:05,567 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:05,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:05,571 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:05,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:05,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,572 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:05,572 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:05,572 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:56:05,572 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:56:05,573 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,573 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,574 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,574 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:05,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:05,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:05,574 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:05,574 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:05,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:05,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:56:05,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:56:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:56:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:56:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:05,575 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:05,575 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. started... 2023-03-08 11:56:05,575 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e53083d9a161a7082c38a975fc8a0192 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:56:05,737 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:56:05,985 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231bee02856303c40b386c84a74e05d777a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231bee02856303c40b386c84a74e05d777a 2023-03-08 11:56:06,391 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/6721bef7608943deb9dd57f0c49f0205 2023-03-08 11:56:06,395 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/6721bef7608943deb9dd57f0c49f0205 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6721bef7608943deb9dd57f0c49f0205 2023-03-08 11:56:06,398 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6721bef7608943deb9dd57f0c49f0205, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 11:56:06,399 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e53083d9a161a7082c38a975fc8a0192 in 824ms, sequenceid=25, compaction requested=true 2023-03-08 11:56:06,399 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:06,399 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:06,399 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:56:06,399 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:56:06,399 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,399 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:56:06,399 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:56:06,401 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,401 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:06,401 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,401 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:06,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:06,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:56:06,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@616926c[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:06,402 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:56:06,402 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,404 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,404 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:56:06,405 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:56:06,405 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:06,405 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,405 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:06,406 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:56:06,406 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,406 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:06,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,406 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:56:06,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:06,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:06,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:06,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:06,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:06,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:06,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:56:06,408 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:56:06,408 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:06,408 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:06,408 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,408 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:06,409 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,409 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:06,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:06,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,409 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:06,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:06,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:06,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:06,410 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:06,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,411 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:06,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,413 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,413 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:06,413 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:06,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:56:06,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:06,413 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:06,413 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:06,413 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,413 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:06,413 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,413 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:06,413 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,413 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:06,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:06,413 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:06,413 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,414 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:06,514 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:06,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:06,518 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:06,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:06,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,520 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:06,520 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:06,520 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:56:06,520 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:56:06,520 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,520 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,523 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,523 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:06,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:06,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:06,523 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:06,523 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:06,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:06,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:56:06,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:06,524 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:56:06,524 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:06,524 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:56:06,524 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:56:06,524 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:06,524 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:06,524 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. started... 2023-03-08 11:56:06,524 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e53083d9a161a7082c38a975fc8a0192 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:56:06,935 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101587c24f583734b78ace1f3c5118f4aef to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101587c24f583734b78ace1f3c5118f4aef 2023-03-08 11:56:07,340 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/326b30784acd407f9de95fcb7b6ba4be 2023-03-08 11:56:07,344 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/326b30784acd407f9de95fcb7b6ba4be as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/326b30784acd407f9de95fcb7b6ba4be 2023-03-08 11:56:07,347 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/326b30784acd407f9de95fcb7b6ba4be, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 11:56:07,347 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e53083d9a161a7082c38a975fc8a0192 in 823ms, sequenceid=29, compaction requested=true 2023-03-08 11:56:07,348 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:07,348 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:07,348 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:56:07,348 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:56:07,348 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,348 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:56:07,348 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:56:07,349 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,349 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:07,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:07,349 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,349 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:07,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:07,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:07,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:07,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:56:07,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26b55737[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:07,351 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:56:07,351 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,352 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:56:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:56:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:07,352 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,352 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:56:07,353 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:56:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:07,355 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:56:07,355 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:56:07,355 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:07,355 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:07,355 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,355 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:07,357 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,357 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:07,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:07,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,357 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:07,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,358 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:07,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:07,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:07,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,359 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,361 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:07,361 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:07,361 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:07,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:07,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:56:07,361 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:07,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:07,461 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:07,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:07,466 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:07,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:07,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,467 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:07,467 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:07,467 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:56:07,467 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:56:07,468 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,468 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,469 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,469 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:07,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:07,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:07,469 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:07,469 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:07,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:07,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,470 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,470 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:56:07,470 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:07,470 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:56:07,470 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:07,470 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:56:07,470 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:56:07,470 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:07,470 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:07,470 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. started... 2023-03-08 11:56:07,471 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e53083d9a161a7082c38a975fc8a0192 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:56:07,881 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034093e1c799084e079ff0c06748893cba to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601034093e1c799084e079ff0c06748893cba 2023-03-08 11:56:08,285 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/c8820cd6f3fb4632b4a2397d1e7f450c 2023-03-08 11:56:08,289 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/c8820cd6f3fb4632b4a2397d1e7f450c as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/c8820cd6f3fb4632b4a2397d1e7f450c 2023-03-08 11:56:08,292 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/c8820cd6f3fb4632b4a2397d1e7f450c, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 11:56:08,293 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e53083d9a161a7082c38a975fc8a0192 in 822ms, sequenceid=33, compaction requested=true 2023-03-08 11:56:08,293 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:08,293 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:08,293 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:56:08,293 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:56:08,293 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,293 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:56:08,293 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:56:08,295 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,295 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:08,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:08,295 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,295 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:08,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:08,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:08,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:08,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:56:08,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c1ed131[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:08,296 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:56:08,296 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,298 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,298 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:56:08,298 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:56:08,298 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:08,298 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,298 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:08,299 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,299 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:56:08,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:08,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:08,300 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:08,300 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:56:08,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:08,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:08,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:08,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:08,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:56:08,301 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:56:08,301 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:08,301 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:08,301 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,301 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:08,305 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,305 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:08,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:08,305 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:08,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:08,306 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:08,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:08,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:08,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,307 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:08,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,308 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,308 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,308 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:08,308 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,309 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:08,309 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:08,309 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,309 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:08,309 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:08,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:08,309 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:08,309 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,309 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,309 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:08,309 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:08,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:08,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:08,409 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:08,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:08,414 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:08,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:08,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,416 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:08,416 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:08,416 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:56:08,416 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:56:08,416 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,416 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,417 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,417 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:08,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:08,417 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,418 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,418 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,419 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:56:08,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,419 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:56:08,419 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:08,419 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:56:08,419 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:56:08,419 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:08,419 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:08,419 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. started... 2023-03-08 11:56:08,419 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e53083d9a161a7082c38a975fc8a0192 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:56:08,833 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907dc3bddddf0354825bd9b1e06187d3ee8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907dc3bddddf0354825bd9b1e06187d3ee8 2023-03-08 11:56:08,839 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/10ff4b7fbc824c20a0098d73fb3fdb57 2023-03-08 11:56:08,842 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/10ff4b7fbc824c20a0098d73fb3fdb57 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/10ff4b7fbc824c20a0098d73fb3fdb57 2023-03-08 11:56:08,845 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/10ff4b7fbc824c20a0098d73fb3fdb57, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 11:56:08,846 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e53083d9a161a7082c38a975fc8a0192 in 427ms, sequenceid=37, compaction requested=true 2023-03-08 11:56:08,846 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:08,846 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:08,846 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:56:08,846 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:56:08,846 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,846 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:56:08,846 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:56:08,848 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,848 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:08,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:08,849 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,849 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:08,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:08,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:08,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:08,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:56:08,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c71589c[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:08,850 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:56:08,850 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,851 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,851 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:56:08,851 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:56:08,851 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,851 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:08,851 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:08,852 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:56:08,852 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:08,852 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,852 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:56:08,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:08,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:08,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:08,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:08,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:08,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:08,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:56:08,855 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:56:08,855 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:08,855 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:08,855 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,855 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:08,856 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,856 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:08,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,856 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:08,857 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:08,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:08,857 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,860 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,860 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,860 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,860 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,860 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:08,860 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,860 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:08,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:08,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:08,860 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:08,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:56:08,860 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:08,861 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:08,861 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,861 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:08,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:08,961 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:08,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:08,965 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:08,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:08,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,967 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:08,967 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:08,967 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 11:56:08,967 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 11:56:08,967 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,968 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,969 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:08,969 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:08,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:08,969 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:08,969 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:08,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:08,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 11:56:08,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:08,970 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 11:56:08,970 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:08,970 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 11:56:08,970 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 11:56:08,970 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:08,970 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:08,970 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. started... 2023-03-08 11:56:08,971 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e53083d9a161a7082c38a975fc8a0192 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 11:56:09,381 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112091cbd595a9fe4b80a3f448ac07251bbb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112091cbd595a9fe4b80a3f448ac07251bbb 2023-03-08 11:56:09,785 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/599e1caab235408ea07d63435ebf1116 2023-03-08 11:56:09,789 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/599e1caab235408ea07d63435ebf1116 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/599e1caab235408ea07d63435ebf1116 2023-03-08 11:56:09,792 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/599e1caab235408ea07d63435ebf1116, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 11:56:09,792 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e53083d9a161a7082c38a975fc8a0192 in 821ms, sequenceid=41, compaction requested=true 2023-03-08 11:56:09,792 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:09,793 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 11:56:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 11:56:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 11:56:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 11:56:09,794 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,794 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:09,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:09,794 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,795 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:09,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:09,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 11:56:09,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ec62e8c[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:09,796 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 11:56:09,796 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,797 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,797 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 11:56:09,797 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 11:56:09,797 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,797 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:09,797 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:09,799 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,799 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 11:56:09,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:09,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:09,799 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:09,799 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 11:56:09,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:09,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 11:56:09,801 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 11:56:09,801 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:09,801 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:09,801 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,801 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:09,802 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,802 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:09,803 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:09,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:09,803 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,804 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:09,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,806 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,806 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:09,806 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,806 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:09,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 11:56:09,806 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 11:56:09,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:09,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:09,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:09,806 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:09,806 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:09,806 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:09,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:09,806 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:09,806 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:09,807 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,807 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,907 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:09,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:09,909 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-08 11:56:09,910 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:56:09,911 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:56:09,911 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:56:09,911 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277169911 2023-03-08 11:56:09,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 11:56:09,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-08 11:56:09,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:56:09,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x41dcee86 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:56:09,917 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@126dfe54, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:56:09,917 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:56:10,740 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010397854a6544804af7bcb17c756dc0e865 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010397854a6544804af7bcb17c756dc0e865 2023-03-08 11:56:10,740 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:56:10,752 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:56:10,755 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57860, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:56:10,760 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:56:10,761 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45446, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:56:10,768 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/4c3ceefa10ec4b6eb35c6c08f8d171b7 first=Optional[r5] last=Optional[r6] 2023-03-08 11:56:10,769 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/4c3ceefa10ec4b6eb35c6c08f8d171b7}] 2023-03-08 11:56:10,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/4c3ceefa10ec4b6eb35c6c08f8d171b7 for inclusion in e53083d9a161a7082c38a975fc8a0192/family1 2023-03-08 11:56:10,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-08 11:56:10,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:56:10,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:10,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/4c3ceefa10ec4b6eb35c6c08f8d171b7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__5utajosaursmu53tn4jq8egivhej0n4ovneeqfkql98e6uqg4so23l3g53c7jia5/family1/4c3ceefa10ec4b6eb35c6c08f8d171b7 2023-03-08 11:56:10,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__5utajosaursmu53tn4jq8egivhej0n4ovneeqfkql98e6uqg4so23l3g53c7jia5/family1/4c3ceefa10ec4b6eb35c6c08f8d171b7 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/3cdbeee795444d67aee54210fbd0a0e4_SeqId_44_ 2023-03-08 11:56:10,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__5utajosaursmu53tn4jq8egivhej0n4ovneeqfkql98e6uqg4so23l3g53c7jia5/family1/4c3ceefa10ec4b6eb35c6c08f8d171b7 into e53083d9a161a7082c38a975fc8a0192/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/3cdbeee795444d67aee54210fbd0a0e4_SeqId_44_ - updating store file list. 2023-03-08 11:56:10,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/3cdbeee795444d67aee54210fbd0a0e4_SeqId_44_ into e53083d9a161a7082c38a975fc8a0192/family1 2023-03-08 11:56:10,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__5utajosaursmu53tn4jq8egivhej0n4ovneeqfkql98e6uqg4so23l3g53c7jia5/family1/4c3ceefa10ec4b6eb35c6c08f8d171b7 into e53083d9a161a7082c38a975fc8a0192/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/3cdbeee795444d67aee54210fbd0a0e4_SeqId_44_) 2023-03-08 11:56:10,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__5utajosaursmu53tn4jq8egivhej0n4ovneeqfkql98e6uqg4so23l3g53c7jia5/family1/4c3ceefa10ec4b6eb35c6c08f8d171b7 2023-03-08 11:56:10,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 11:56:10,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 11:56:10,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): e53083d9a161a7082c38a975fc8a0192/family1 is initiating minor compaction 2023-03-08 11:56:10,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@53eb6a28; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:56:10,787 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e53083d9a161a7082c38a975fc8a0192/family1 in testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:10,788 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/8987500102bb438390f1a6b6033873a6, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/94bc870370844b52b7fb97e6698974be, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6f843cbbd2364d6a97ceae59eeb04322, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/da54dfc5e5d84809bb803e654a48cc46, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6721bef7608943deb9dd57f0c49f0205, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/326b30784acd407f9de95fcb7b6ba4be, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/c8820cd6f3fb4632b4a2397d1e7f450c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/10ff4b7fbc824c20a0098d73fb3fdb57, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/599e1caab235408ea07d63435ebf1116, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/3cdbeee795444d67aee54210fbd0a0e4_SeqId_44_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp, totalSize=49.7 K 2023-03-08 11:56:10,788 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 8987500102bb438390f1a6b6033873a6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 11:56:10,788 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 94bc870370844b52b7fb97e6698974be, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 11:56:10,788 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 6f843cbbd2364d6a97ceae59eeb04322, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 11:56:10,788 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting da54dfc5e5d84809bb803e654a48cc46, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 11:56:10,788 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 6721bef7608943deb9dd57f0c49f0205, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 11:56:10,788 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 326b30784acd407f9de95fcb7b6ba4be, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 11:56:10,788 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting c8820cd6f3fb4632b4a2397d1e7f450c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 11:56:10,788 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 10ff4b7fbc824c20a0098d73fb3fdb57, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 11:56:10,788 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 599e1caab235408ea07d63435ebf1116, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 11:56:10,788 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting 3cdbeee795444d67aee54210fbd0a0e4_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 11:56:10,789 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:56:10,790 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:10,792 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101587c24f583734b78ace1f3c5118f4aef to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101587c24f583734b78ace1f3c5118f4aef 2023-03-08 11:56:10,793 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601034093e1c799084e079ff0c06748893cba to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601034093e1c799084e079ff0c06748893cba 2023-03-08 11:56:10,793 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-08 11:56:10,793 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-08 11:56:10,794 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:56:10,794 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:56:10,824 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): e53083d9a161a7082c38a975fc8a0192#family1#compaction#164 average throughput is 0.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:56:10,839 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/a97dea9f0a834ddf9b15f58beac0d873 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/a97dea9f0a834ddf9b15f58beac0d873 2023-03-08 11:56:10,842 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in e53083d9a161a7082c38a975fc8a0192/family1 of e53083d9a161a7082c38a975fc8a0192 into a97dea9f0a834ddf9b15f58beac0d873(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:56:10,842 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:10,842 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678276570787; duration=0sec 2023-03-08 11:56:10,842 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:56:10,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/8987500102bb438390f1a6b6033873a6, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/94bc870370844b52b7fb97e6698974be, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6f843cbbd2364d6a97ceae59eeb04322, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/da54dfc5e5d84809bb803e654a48cc46, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6721bef7608943deb9dd57f0c49f0205, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/326b30784acd407f9de95fcb7b6ba4be, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/c8820cd6f3fb4632b4a2397d1e7f450c, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/10ff4b7fbc824c20a0098d73fb3fdb57, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/599e1caab235408ea07d63435ebf1116, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/3cdbeee795444d67aee54210fbd0a0e4_SeqId_44_] to archive 2023-03-08 11:56:10,874 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:10,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/8987500102bb438390f1a6b6033873a6 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/8987500102bb438390f1a6b6033873a6 2023-03-08 11:56:10,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/94bc870370844b52b7fb97e6698974be to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/94bc870370844b52b7fb97e6698974be 2023-03-08 11:56:10,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6f843cbbd2364d6a97ceae59eeb04322 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6f843cbbd2364d6a97ceae59eeb04322 2023-03-08 11:56:10,878 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/da54dfc5e5d84809bb803e654a48cc46 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/da54dfc5e5d84809bb803e654a48cc46 2023-03-08 11:56:10,879 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6721bef7608943deb9dd57f0c49f0205 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/6721bef7608943deb9dd57f0c49f0205 2023-03-08 11:56:10,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/326b30784acd407f9de95fcb7b6ba4be to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/326b30784acd407f9de95fcb7b6ba4be 2023-03-08 11:56:10,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/c8820cd6f3fb4632b4a2397d1e7f450c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/c8820cd6f3fb4632b4a2397d1e7f450c 2023-03-08 11:56:10,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/10ff4b7fbc824c20a0098d73fb3fdb57 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/10ff4b7fbc824c20a0098d73fb3fdb57 2023-03-08 11:56:10,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/599e1caab235408ea07d63435ebf1116 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/599e1caab235408ea07d63435ebf1116 2023-03-08 11:56:10,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/3cdbeee795444d67aee54210fbd0a0e4_SeqId_44_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/3cdbeee795444d67aee54210fbd0a0e4_SeqId_44_ 2023-03-08 11:56:11,633 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123126fa7b001f7c4235a824f6024ea27634 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123126fa7b001f7c4235a824f6024ea27634 2023-03-08 11:56:11,634 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:56:11,657 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/40ac1c624c9b4b8eaea2877289d1d140 first=Optional[r1] last=Optional[r4] 2023-03-08 11:56:11,658 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/40ac1c624c9b4b8eaea2877289d1d140}] 2023-03-08 11:56:11,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/40ac1c624c9b4b8eaea2877289d1d140 for inclusion in e53083d9a161a7082c38a975fc8a0192/family1 2023-03-08 11:56:11,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-08 11:56:11,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:56:11,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:11,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/40ac1c624c9b4b8eaea2877289d1d140 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__kgv01lngjveto8inq0ouoatep7uonql5t7vhv0pss51q25iuv0bas52qulvgeql3/family1/40ac1c624c9b4b8eaea2877289d1d140 2023-03-08 11:56:11,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__kgv01lngjveto8inq0ouoatep7uonql5t7vhv0pss51q25iuv0bas52qulvgeql3/family1/40ac1c624c9b4b8eaea2877289d1d140 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/f5df5adca6d74cc9977970b6aa55f6a0_SeqId_47_ 2023-03-08 11:56:11,666 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__kgv01lngjveto8inq0ouoatep7uonql5t7vhv0pss51q25iuv0bas52qulvgeql3/family1/40ac1c624c9b4b8eaea2877289d1d140 into e53083d9a161a7082c38a975fc8a0192/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/f5df5adca6d74cc9977970b6aa55f6a0_SeqId_47_ - updating store file list. 2023-03-08 11:56:11,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/f5df5adca6d74cc9977970b6aa55f6a0_SeqId_47_ into e53083d9a161a7082c38a975fc8a0192/family1 2023-03-08 11:56:11,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__kgv01lngjveto8inq0ouoatep7uonql5t7vhv0pss51q25iuv0bas52qulvgeql3/family1/40ac1c624c9b4b8eaea2877289d1d140 into e53083d9a161a7082c38a975fc8a0192/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/f5df5adca6d74cc9977970b6aa55f6a0_SeqId_47_) 2023-03-08 11:56:11,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__kgv01lngjveto8inq0ouoatep7uonql5t7vhv0pss51q25iuv0bas52qulvgeql3/family1/40ac1c624c9b4b8eaea2877289d1d140 2023-03-08 11:56:11,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 11:56:11,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 11:56:11,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(1912): e53083d9a161a7082c38a975fc8a0192/family1 is initiating minor compaction (all files) 2023-03-08 11:56:11,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@24261b0b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:56:11,671 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e53083d9a161a7082c38a975fc8a0192/family1 in testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:11,671 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/dc3f445526154aff9c80a3408277dd71, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/a97dea9f0a834ddf9b15f58beac0d873, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/f5df5adca6d74cc9977970b6aa55f6a0_SeqId_47_] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp, totalSize=178.1 K 2023-03-08 11:56:11,671 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting dc3f445526154aff9c80a3408277dd71, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838541 2023-03-08 11:56:11,672 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting a97dea9f0a834ddf9b15f58beac0d873, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238541 2023-03-08 11:56:11,672 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] compactions.Compactor(207): Compacting f5df5adca6d74cc9977970b6aa55f6a0_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965238541 2023-03-08 11:56:11,673 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:56:11,673 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:11,675 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201ec69519ca7ee4da9a4073de5ed7c89ba to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201ec69519ca7ee4da9a4073de5ed7c89ba 2023-03-08 11:56:11,676 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512054ceb5e24eb414013abb954cc8f3631e8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512054ceb5e24eb414013abb954cc8f3631e8 2023-03-08 11:56:11,677 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512286141dc9d2d764eab8b59576ad297143a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512286141dc9d2d764eab8b59576ad297143a 2023-03-08 11:56:11,678 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231bee02856303c40b386c84a74e05d777a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231bee02856303c40b386c84a74e05d777a 2023-03-08 11:56:11,679 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-08 11:56:11,679 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-08 11:56:11,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:56:11,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:56:11,685 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] throttle.PressureAwareThroughputController(145): e53083d9a161a7082c38a975fc8a0192#family1#compaction#165 average throughput is 25.93 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:56:12,123 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130e444c8295af64d02a94c0b9a73ab3b93 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130e444c8295af64d02a94c0b9a73ab3b93 2023-03-08 11:56:12,123 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:56:12,147 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3fd7ecbf1fd844d8a9a2bf3c9415ebc4 first=Optional[r00] last=Optional[r8] 2023-03-08 11:56:12,147 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3fd7ecbf1fd844d8a9a2bf3c9415ebc4}] 2023-03-08 11:56:12,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3fd7ecbf1fd844d8a9a2bf3c9415ebc4 for inclusion in e53083d9a161a7082c38a975fc8a0192/family1 2023-03-08 11:56:12,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-08 11:56:12,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 11:56:12,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:12,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3fd7ecbf1fd844d8a9a2bf3c9415ebc4 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__b7afvg6n84mth89aktgnq9o6obldf5i8pj04ovhq6rq5c1eqprqq5quch5lrpofs/family1/3fd7ecbf1fd844d8a9a2bf3c9415ebc4 2023-03-08 11:56:12,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__b7afvg6n84mth89aktgnq9o6obldf5i8pj04ovhq6rq5c1eqprqq5quch5lrpofs/family1/3fd7ecbf1fd844d8a9a2bf3c9415ebc4 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/00cd57c3f1c44c12bd931c7df2fcadc3_SeqId_49_ 2023-03-08 11:56:12,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__b7afvg6n84mth89aktgnq9o6obldf5i8pj04ovhq6rq5c1eqprqq5quch5lrpofs/family1/3fd7ecbf1fd844d8a9a2bf3c9415ebc4 into e53083d9a161a7082c38a975fc8a0192/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/00cd57c3f1c44c12bd931c7df2fcadc3_SeqId_49_ - updating store file list. 2023-03-08 11:56:12,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/00cd57c3f1c44c12bd931c7df2fcadc3_SeqId_49_ into e53083d9a161a7082c38a975fc8a0192/family1 2023-03-08 11:56:12,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__b7afvg6n84mth89aktgnq9o6obldf5i8pj04ovhq6rq5c1eqprqq5quch5lrpofs/family1/3fd7ecbf1fd844d8a9a2bf3c9415ebc4 into e53083d9a161a7082c38a975fc8a0192/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/00cd57c3f1c44c12bd931c7df2fcadc3_SeqId_49_) 2023-03-08 11:56:12,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__b7afvg6n84mth89aktgnq9o6obldf5i8pj04ovhq6rq5c1eqprqq5quch5lrpofs/family1/3fd7ecbf1fd844d8a9a2bf3c9415ebc4 2023-03-08 11:56:12,158 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:56:12,159 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:12,160 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112091cbd595a9fe4b80a3f448ac07251bbb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112091cbd595a9fe4b80a3f448ac07251bbb 2023-03-08 11:56:12,161 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112814748d6d963f4ae489020cf9769f4296 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112814748d6d963f4ae489020cf9769f4296 2023-03-08 11:56:12,162 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511301d75aef396fa4c409ba432824a9cc4bd to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511301d75aef396fa4c409ba432824a9cc4bd 2023-03-08 11:56:12,162 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-08 11:56:12,162 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 11:56:12,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:56:12,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:56:12,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 11:56:12,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 11:56:12,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:56:12,164 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 11:56:12,165 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.2550 sec 2023-03-08 11:56:12,346 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=255 (was 254) - Thread LEAK? -, OpenFileDescriptor=642 (was 638) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=155 (was 150) - SystemLoadAverage LEAK? -, ProcessCount=173 (was 173), AvailableMemoryMB=7343 (was 7365) 2023-03-08 11:56:12,357 INFO [Listener at localhost/36211] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=255, OpenFileDescriptor=642, MaxFileDescriptor=60000, SystemLoadAverage=155, ProcessCount=173, AvailableMemoryMB=7342 2023-03-08 11:56:12,376 DEBUG [Listener at localhost/36211] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-08 11:56:12,441 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10r\xB6\xD9v\xAB\xE3\xBE\x0C_\x94+\xDBA\x02\x02G"\x10\xB3\xB5\xEB:"2\xE3k\x88\xF9\x16G'*,\x15*\x10\xC4\x82a\xC2d\xE2\xB7\xA3\x8DL\x12\xBE&\xE7xG2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 11:56:12,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-08 11:56:12,443 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 11:56:12,444 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-08 11:56:12,444 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 11:56:12,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 11:56:12,446 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:12,446 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:12,446 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:12,446 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3 empty. 2023-03-08 11:56:12,446 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610 empty. 2023-03-08 11:56:12,446 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0 empty. 2023-03-08 11:56:12,446 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:12,446 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:12,446 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:12,447 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-08 11:56:12,512 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/.tmp/family1/5d284c5b96e6406a89ba6fc294a31e3b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/5d284c5b96e6406a89ba6fc294a31e3b 2023-03-08 11:56:12,516 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in e53083d9a161a7082c38a975fc8a0192/family1 of e53083d9a161a7082c38a975fc8a0192 into 5d284c5b96e6406a89ba6fc294a31e3b(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 11:56:12,516 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:12,516 INFO [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678276571671; duration=0sec 2023-03-08 11:56:12,516 DEBUG [RS:0;jenkins-hbase4:45551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 11:56:12,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 11:56:12,572 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/dc3f445526154aff9c80a3408277dd71, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/a97dea9f0a834ddf9b15f58beac0d873, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/f5df5adca6d74cc9977970b6aa55f6a0_SeqId_47_] to archive 2023-03-08 11:56:12,573 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:12,574 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/dc3f445526154aff9c80a3408277dd71 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/dc3f445526154aff9c80a3408277dd71 2023-03-08 11:56:12,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/a97dea9f0a834ddf9b15f58beac0d873 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/a97dea9f0a834ddf9b15f58beac0d873 2023-03-08 11:56:12,576 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/f5df5adca6d74cc9977970b6aa55f6a0_SeqId_47_ to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/family1/f5df5adca6d74cc9977970b6aa55f6a0_SeqId_47_ 2023-03-08 11:56:12,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 11:56:12,853 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-08 11:56:12,855 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 30ea0c03bb5d1a1afdaa8ef87fc4f610, NAME => 'testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10r\xB6\xD9v\xAB\xE3\xBE\x0C_\x94+\xDBA\x02\x02G"\x10\xB3\xB5\xEB:"2\xE3k\x88\xF9\x16G'*,\x15*\x10\xC4\x82a\xC2d\xE2\xB7\xA3\x8DL\x12\xBE&\xE7xG2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:56:12,855 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => ff22c265b3d4c9370e800d08d19c30b0, NAME => 'testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10r\xB6\xD9v\xAB\xE3\xBE\x0C_\x94+\xDBA\x02\x02G"\x10\xB3\xB5\xEB:"2\xE3k\x88\xF9\x16G'*,\x15*\x10\xC4\x82a\xC2d\xE2\xB7\xA3\x8DL\x12\xBE&\xE7xG2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:56:12,855 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => ffda891a90e48be952988d132e357ad3, NAME => 'testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10r\xB6\xD9v\xAB\xE3\xBE\x0C_\x94+\xDBA\x02\x02G"\x10\xB3\xB5\xEB:"2\xE3k\x88\xF9\x16G'*,\x15*\x10\xC4\x82a\xC2d\xE2\xB7\xA3\x8DL\x12\xBE&\xE7xG2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/.tmp 2023-03-08 11:56:12,869 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:56:12,869 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing ff22c265b3d4c9370e800d08d19c30b0, disabling compactions & flushes 2023-03-08 11:56:12,869 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:12,869 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:12,869 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. after waiting 0 ms 2023-03-08 11:56:12,869 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:12,869 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:12,869 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:13,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 11:56:13,269 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:56:13,269 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 30ea0c03bb5d1a1afdaa8ef87fc4f610, disabling compactions & flushes 2023-03-08 11:56:13,269 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:13,269 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:13,269 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:56:13,269 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. after waiting 0 ms 2023-03-08 11:56:13,269 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing ffda891a90e48be952988d132e357ad3, disabling compactions & flushes 2023-03-08 11:56:13,269 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:13,269 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:13,269 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:13,269 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:13,270 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. after waiting 0 ms 2023-03-08 11:56:13,270 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:13,270 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:13,270 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:13,270 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:13,272 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 11:56:13,272 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678276573272"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276573272"}]},"ts":"1678276573272"} 2023-03-08 11:56:13,273 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276573272"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276573272"}]},"ts":"1678276573272"} 2023-03-08 11:56:13,273 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276573272"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678276573272"}]},"ts":"1678276573272"} 2023-03-08 11:56:13,274 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 11:56:13,275 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 11:56:13,275 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276573275"}]},"ts":"1678276573275"} 2023-03-08 11:56:13,276 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-08 11:56:13,279 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ffda891a90e48be952988d132e357ad3, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ff22c265b3d4c9370e800d08d19c30b0, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30ea0c03bb5d1a1afdaa8ef87fc4f610, ASSIGN}] 2023-03-08 11:56:13,281 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30ea0c03bb5d1a1afdaa8ef87fc4f610, ASSIGN 2023-03-08 11:56:13,281 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ffda891a90e48be952988d132e357ad3, ASSIGN 2023-03-08 11:56:13,281 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ff22c265b3d4c9370e800d08d19c30b0, ASSIGN 2023-03-08 11:56:13,282 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ffda891a90e48be952988d132e357ad3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:56:13,282 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30ea0c03bb5d1a1afdaa8ef87fc4f610, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:56:13,282 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ff22c265b3d4c9370e800d08d19c30b0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45551,1678276414946; forceNewPlan=false, retain=false 2023-03-08 11:56:13,434 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=ff22c265b3d4c9370e800d08d19c30b0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:13,434 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=ffda891a90e48be952988d132e357ad3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:13,434 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=30ea0c03bb5d1a1afdaa8ef87fc4f610, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:13,434 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678276573434"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276573434"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276573434"}]},"ts":"1678276573434"} 2023-03-08 11:56:13,434 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276573434"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276573434"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276573434"}]},"ts":"1678276573434"} 2023-03-08 11:56:13,435 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276573434"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678276573434"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678276573434"}]},"ts":"1678276573434"} 2023-03-08 11:56:13,436 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=118, state=RUNNABLE; OpenRegionProcedure ff22c265b3d4c9370e800d08d19c30b0, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:56:13,436 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=117, state=RUNNABLE; OpenRegionProcedure ffda891a90e48be952988d132e357ad3, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:56:13,437 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure 30ea0c03bb5d1a1afdaa8ef87fc4f610, server=jenkins-hbase4.apache.org,45551,1678276414946}] 2023-03-08 11:56:13,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 11:56:13,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:13,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ff22c265b3d4c9370e800d08d19c30b0, NAME => 'testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 11:56:13,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:13,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:56:13,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:13,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:13,611 INFO [StoreOpener-ff22c265b3d4c9370e800d08d19c30b0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:13,613 DEBUG [StoreOpener-ff22c265b3d4c9370e800d08d19c30b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1 2023-03-08 11:56:13,613 DEBUG [StoreOpener-ff22c265b3d4c9370e800d08d19c30b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1 2023-03-08 11:56:13,613 INFO [StoreOpener-ff22c265b3d4c9370e800d08d19c30b0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff22c265b3d4c9370e800d08d19c30b0 columnFamilyName family1 2023-03-08 11:56:13,614 INFO [StoreOpener-ff22c265b3d4c9370e800d08d19c30b0-1] regionserver.HStore(310): Store=ff22c265b3d4c9370e800d08d19c30b0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:56:13,615 INFO [StoreOpener-ff22c265b3d4c9370e800d08d19c30b0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:13,616 DEBUG [StoreOpener-ff22c265b3d4c9370e800d08d19c30b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2 2023-03-08 11:56:13,616 DEBUG [StoreOpener-ff22c265b3d4c9370e800d08d19c30b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2 2023-03-08 11:56:13,616 INFO [StoreOpener-ff22c265b3d4c9370e800d08d19c30b0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff22c265b3d4c9370e800d08d19c30b0 columnFamilyName family2 2023-03-08 11:56:13,617 INFO [StoreOpener-ff22c265b3d4c9370e800d08d19c30b0-1] regionserver.HStore(310): Store=ff22c265b3d4c9370e800d08d19c30b0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:56:13,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:13,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:13,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:56:13,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:13,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:56:13,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ff22c265b3d4c9370e800d08d19c30b0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70116820, jitterRate=0.04482203722000122}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:56:13,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:13,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0., pid=120, masterSystemTime=1678276573587 2023-03-08 11:56:13,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:13,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:13,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:13,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ffda891a90e48be952988d132e357ad3, NAME => 'testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 11:56:13,624 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=ff22c265b3d4c9370e800d08d19c30b0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:13,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:13,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:56:13,624 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678276573624"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276573624"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276573624"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276573624"}]},"ts":"1678276573624"} 2023-03-08 11:56:13,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:13,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:13,627 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=118 2023-03-08 11:56:13,627 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=118, state=SUCCESS; OpenRegionProcedure ff22c265b3d4c9370e800d08d19c30b0, server=jenkins-hbase4.apache.org,45551,1678276414946 in 189 msec 2023-03-08 11:56:13,627 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ff22c265b3d4c9370e800d08d19c30b0, ASSIGN in 348 msec 2023-03-08 11:56:13,644 INFO [StoreOpener-ffda891a90e48be952988d132e357ad3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:13,645 DEBUG [StoreOpener-ffda891a90e48be952988d132e357ad3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1 2023-03-08 11:56:13,645 DEBUG [StoreOpener-ffda891a90e48be952988d132e357ad3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1 2023-03-08 11:56:13,646 INFO [StoreOpener-ffda891a90e48be952988d132e357ad3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ffda891a90e48be952988d132e357ad3 columnFamilyName family1 2023-03-08 11:56:13,646 INFO [StoreOpener-ffda891a90e48be952988d132e357ad3-1] regionserver.HStore(310): Store=ffda891a90e48be952988d132e357ad3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:56:13,647 INFO [StoreOpener-ffda891a90e48be952988d132e357ad3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:13,647 DEBUG [StoreOpener-ffda891a90e48be952988d132e357ad3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2 2023-03-08 11:56:13,647 DEBUG [StoreOpener-ffda891a90e48be952988d132e357ad3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2 2023-03-08 11:56:13,648 INFO [StoreOpener-ffda891a90e48be952988d132e357ad3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ffda891a90e48be952988d132e357ad3 columnFamilyName family2 2023-03-08 11:56:13,648 INFO [StoreOpener-ffda891a90e48be952988d132e357ad3-1] regionserver.HStore(310): Store=ffda891a90e48be952988d132e357ad3/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:56:13,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:13,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:13,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:56:13,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:13,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:56:13,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ffda891a90e48be952988d132e357ad3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73170901, jitterRate=0.09033139050006866}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:56:13,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:13,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3., pid=121, masterSystemTime=1678276573587 2023-03-08 11:56:13,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:13,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:13,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:13,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 30ea0c03bb5d1a1afdaa8ef87fc4f610, NAME => 'testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 11:56:13,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:13,655 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=ffda891a90e48be952988d132e357ad3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:13,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 11:56:13,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:13,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:13,655 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276573654"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276573654"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276573654"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276573654"}]},"ts":"1678276573654"} 2023-03-08 11:56:13,657 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=117 2023-03-08 11:56:13,657 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=117, state=SUCCESS; OpenRegionProcedure ffda891a90e48be952988d132e357ad3, server=jenkins-hbase4.apache.org,45551,1678276414946 in 220 msec 2023-03-08 11:56:13,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ffda891a90e48be952988d132e357ad3, ASSIGN in 378 msec 2023-03-08 11:56:13,675 INFO [StoreOpener-30ea0c03bb5d1a1afdaa8ef87fc4f610-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:13,676 DEBUG [StoreOpener-30ea0c03bb5d1a1afdaa8ef87fc4f610-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1 2023-03-08 11:56:13,676 DEBUG [StoreOpener-30ea0c03bb5d1a1afdaa8ef87fc4f610-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1 2023-03-08 11:56:13,676 INFO [StoreOpener-30ea0c03bb5d1a1afdaa8ef87fc4f610-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30ea0c03bb5d1a1afdaa8ef87fc4f610 columnFamilyName family1 2023-03-08 11:56:13,677 INFO [StoreOpener-30ea0c03bb5d1a1afdaa8ef87fc4f610-1] regionserver.HStore(310): Store=30ea0c03bb5d1a1afdaa8ef87fc4f610/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:56:13,677 INFO [StoreOpener-30ea0c03bb5d1a1afdaa8ef87fc4f610-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:13,678 DEBUG [StoreOpener-30ea0c03bb5d1a1afdaa8ef87fc4f610-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2 2023-03-08 11:56:13,678 DEBUG [StoreOpener-30ea0c03bb5d1a1afdaa8ef87fc4f610-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2 2023-03-08 11:56:13,678 INFO [StoreOpener-30ea0c03bb5d1a1afdaa8ef87fc4f610-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30ea0c03bb5d1a1afdaa8ef87fc4f610 columnFamilyName family2 2023-03-08 11:56:13,679 INFO [StoreOpener-30ea0c03bb5d1a1afdaa8ef87fc4f610-1] regionserver.HStore(310): Store=30ea0c03bb5d1a1afdaa8ef87fc4f610/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 11:56:13,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:13,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:13,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 11:56:13,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:13,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 11:56:13,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 30ea0c03bb5d1a1afdaa8ef87fc4f610; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74123833, jitterRate=0.1045311838388443}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 11:56:13,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:13,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610., pid=122, masterSystemTime=1678276573587 2023-03-08 11:56:13,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:13,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:13,685 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=30ea0c03bb5d1a1afdaa8ef87fc4f610, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:13,685 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678276573685"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678276573685"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678276573685"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678276573685"}]},"ts":"1678276573685"} 2023-03-08 11:56:13,687 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-03-08 11:56:13,687 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure 30ea0c03bb5d1a1afdaa8ef87fc4f610, server=jenkins-hbase4.apache.org,45551,1678276414946 in 249 msec 2023-03-08 11:56:13,688 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-03-08 11:56:13,688 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30ea0c03bb5d1a1afdaa8ef87fc4f610, ASSIGN in 408 msec 2023-03-08 11:56:13,689 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 11:56:13,689 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678276573689"}]},"ts":"1678276573689"} 2023-03-08 11:56:13,690 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-08 11:56:13,691 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 11:56:13,692 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2500 sec 2023-03-08 11:56:14,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 11:56:14,549 INFO [Listener at localhost/36211] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-08 11:56:14,554 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:14,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:14,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:14,557 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:14,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 11:56:14,563 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 11:56:14,563 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:14,563 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 11:56:14,563 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 11:56:14,564 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:14,564 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:14,568 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:14,568 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:14,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:14,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:14,568 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:14,568 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:14,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:14,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:14,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:14,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:14,569 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 11:56:14,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 11:56:14,569 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 11:56:14,569 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:14,570 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 11:56:14,570 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:56:14,570 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:14,570 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:14,570 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:14,570 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. started... 2023-03-08 11:56:14,570 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. started... 2023-03-08 11:56:14,570 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. started... 2023-03-08 11:56:14,570 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:14,571 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 30ea0c03bb5d1a1afdaa8ef87fc4f610 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 11:56:14,571 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ffda891a90e48be952988d132e357ad3 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 11:56:14,571 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff22c265b3d4c9370e800d08d19c30b0 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 11:56:14,619 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308ce30166840a143b6897b6d2ec4647016 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308ce30166840a143b6897b6d2ec4647016 2023-03-08 11:56:15,020 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308b5a29edef35a44d699b9207d1c581692 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308b5a29edef35a44d699b9207d1c581692 2023-03-08 11:56:15,020 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303087a0b85431a3e4749ba6a7d7aaafa8060 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303087a0b85431a3e4749ba6a7d7aaafa8060 2023-03-08 11:56:15,034 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/bf9c9528256f4e72a4bab498d62a87b7 2023-03-08 11:56:15,435 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/e68b4380849049da813135482a908328 2023-03-08 11:56:15,440 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/e3dbcae5d745483f8af5f980d25e26f1 2023-03-08 11:56:15,468 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308545e11aa0e29499480ec5531ed14dd52 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308545e11aa0e29499480ec5531ed14dd52 2023-03-08 11:56:15,473 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303086007d08be67d416bb47ac05159306c28 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303086007d08be67d416bb47ac05159306c28 2023-03-08 11:56:15,485 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308f937bc70a926438795db9199d6ed2bb3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308f937bc70a926438795db9199d6ed2bb3 2023-03-08 11:56:15,489 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/599c0c789b0448469e0773c0801a733b 2023-03-08 11:56:15,493 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/e3dbcae5d745483f8af5f980d25e26f1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/e3dbcae5d745483f8af5f980d25e26f1 2023-03-08 11:56:15,516 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/e3dbcae5d745483f8af5f980d25e26f1, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 11:56:15,517 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/599c0c789b0448469e0773c0801a733b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/599c0c789b0448469e0773c0801a733b 2023-03-08 11:56:15,520 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/599c0c789b0448469e0773c0801a733b, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 11:56:15,521 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 30ea0c03bb5d1a1afdaa8ef87fc4f610 in 950ms, sequenceid=5, compaction requested=false 2023-03-08 11:56:15,521 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-08 11:56:15,521 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:15,521 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:15,521 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:56:15,879 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/8e40a73f66b440ac9e0a47c3ecf0fac7 2023-03-08 11:56:15,879 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/7e58d655a9674f788d12c3fbcf88d537 2023-03-08 11:56:15,882 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/e68b4380849049da813135482a908328 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/e68b4380849049da813135482a908328 2023-03-08 11:56:15,883 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/bf9c9528256f4e72a4bab498d62a87b7 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/bf9c9528256f4e72a4bab498d62a87b7 2023-03-08 11:56:15,905 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/e68b4380849049da813135482a908328, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 11:56:15,905 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/bf9c9528256f4e72a4bab498d62a87b7, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 11:56:15,906 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/8e40a73f66b440ac9e0a47c3ecf0fac7 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/8e40a73f66b440ac9e0a47c3ecf0fac7 2023-03-08 11:56:15,906 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/7e58d655a9674f788d12c3fbcf88d537 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/7e58d655a9674f788d12c3fbcf88d537 2023-03-08 11:56:15,909 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/8e40a73f66b440ac9e0a47c3ecf0fac7, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 11:56:15,910 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/7e58d655a9674f788d12c3fbcf88d537, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 11:56:15,910 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for ff22c265b3d4c9370e800d08d19c30b0 in 1339ms, sequenceid=5, compaction requested=false 2023-03-08 11:56:15,910 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for ffda891a90e48be952988d132e357ad3 in 1339ms, sequenceid=5, compaction requested=false 2023-03-08 11:56:15,911 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:15,911 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:15,911 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:15,911 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:15,911 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:56:15,911 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:56:15,911 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:56:15,911 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,912 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 11:56:15,912 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 11:56:15,913 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,913 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:15,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:15,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:15,913 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:15,913 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:15,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:15,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:15,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:15,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:15,915 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 11:56:15,915 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3246ca8c[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:15,915 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 11:56:15,915 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:15,916 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:15,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:15,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:15,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 11:56:15,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 11:56:15,916 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:15,916 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,916 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:15,918 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,918 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 11:56:15,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:15,918 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:15,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:15,919 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 11:56:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:15,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 11:56:15,921 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 11:56:15,921 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:15,921 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:15,921 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 11:56:15,921 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:15,922 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:15,922 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:15,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:15,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:15,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:15,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:15,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:15,922 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:15,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:15,923 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:15,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:15,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:15,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:15,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:15,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:15,924 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:15,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:15,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,926 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,926 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:15,926 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:15,926 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 11:56:15,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 11:56:15,926 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:15,926 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:15,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:15,926 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:15,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:15,926 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:15,927 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:15,927 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:15,927 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:15,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:15,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:15,927 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:15,927 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:15,927 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:16,027 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:16,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:16,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:16,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:16,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:16,031 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:16,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:16,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 11:56:16,033 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 11:56:16,033 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:16,033 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 11:56:16,033 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 11:56:16,034 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:16,034 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:16,035 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:16,035 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:16,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:16,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:16,035 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:16,035 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:16,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:16,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:16,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 11:56:16,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 11:56:16,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 11:56:16,036 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 11:56:16,036 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:16,036 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 11:56:16,037 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:16,037 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:56:16,037 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:16,037 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:16,037 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. started... 2023-03-08 11:56:16,037 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. started... 2023-03-08 11:56:16,037 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:16,037 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. started... 2023-03-08 11:56:16,037 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff22c265b3d4c9370e800d08d19c30b0 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 11:56:16,037 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ffda891a90e48be952988d132e357ad3 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 11:56:16,037 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 30ea0c03bb5d1a1afdaa8ef87fc4f610 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 11:56:16,082 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030890ae4eb634d744898a66004cc40c17f3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030890ae4eb634d744898a66004cc40c17f3 2023-03-08 11:56:16,096 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/34b929af3c9841dcb47922b43ecaf397 2023-03-08 11:56:16,103 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308857bbc733d0d46ada2c1d363512d216c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308857bbc733d0d46ada2c1d363512d216c 2023-03-08 11:56:16,492 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030884b3806a33f8476c82663e06ef7005fb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030884b3806a33f8476c82663e06ef7005fb 2023-03-08 11:56:16,517 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/ef424c2cff2f4557b30bf138d3d4f8cf 2023-03-08 11:56:16,539 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308bd7ac639fe134868878754a150154720 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308bd7ac639fe134868878754a150154720 2023-03-08 11:56:16,906 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/55fda5e064f24c9390fa240e6941174f 2023-03-08 11:56:16,944 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/c25e3152893344dc9dc57618d6325f41 2023-03-08 11:56:16,949 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/34b929af3c9841dcb47922b43ecaf397 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/34b929af3c9841dcb47922b43ecaf397 2023-03-08 11:56:16,959 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308e2f87506c050421d9f506eafc3d7ea4b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230308e2f87506c050421d9f506eafc3d7ea4b 2023-03-08 11:56:16,964 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303082477ab66eeda49cc98cc7ce70a776368 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303082477ab66eeda49cc98cc7ce70a776368 2023-03-08 11:56:16,976 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/34b929af3c9841dcb47922b43ecaf397, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 11:56:16,977 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/c25e3152893344dc9dc57618d6325f41 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/c25e3152893344dc9dc57618d6325f41 2023-03-08 11:56:16,981 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/c25e3152893344dc9dc57618d6325f41, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 11:56:16,982 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for ff22c265b3d4c9370e800d08d19c30b0 in 945ms, sequenceid=9, compaction requested=false 2023-03-08 11:56:16,982 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:16,982 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:16,982 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:56:17,374 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/cc7faf7b68144d0aafd1b2941f93f97f 2023-03-08 11:56:17,375 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/d18e572dcf704bcca507adb57c00b124 2023-03-08 11:56:17,379 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/ef424c2cff2f4557b30bf138d3d4f8cf as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/ef424c2cff2f4557b30bf138d3d4f8cf 2023-03-08 11:56:17,381 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/55fda5e064f24c9390fa240e6941174f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/55fda5e064f24c9390fa240e6941174f 2023-03-08 11:56:17,405 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/ef424c2cff2f4557b30bf138d3d4f8cf, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 11:56:17,406 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/55fda5e064f24c9390fa240e6941174f, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 11:56:17,406 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/cc7faf7b68144d0aafd1b2941f93f97f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/cc7faf7b68144d0aafd1b2941f93f97f 2023-03-08 11:56:17,407 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/d18e572dcf704bcca507adb57c00b124 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/d18e572dcf704bcca507adb57c00b124 2023-03-08 11:56:17,410 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/cc7faf7b68144d0aafd1b2941f93f97f, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 11:56:17,411 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/d18e572dcf704bcca507adb57c00b124, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 11:56:17,411 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for ffda891a90e48be952988d132e357ad3 in 1374ms, sequenceid=9, compaction requested=false 2023-03-08 11:56:17,411 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:17,411 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:17,411 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:56:17,411 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 30ea0c03bb5d1a1afdaa8ef87fc4f610 in 1374ms, sequenceid=9, compaction requested=false 2023-03-08 11:56:17,412 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:17,412 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:17,412 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:56:17,412 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:56:17,412 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,412 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 11:56:17,412 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 11:56:17,413 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:17,413 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:17,414 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:17,414 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:17,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:17,415 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 11:56:17,415 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@385b12ac[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:17,415 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 11:56:17,415 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:17,416 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:17,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:17,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:17,416 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 11:56:17,416 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 11:56:17,416 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:17,416 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,416 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:17,417 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 11:56:17,417 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,417 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:17,417 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 11:56:17,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:17,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:17,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:17,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:17,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:17,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:17,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:17,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:17,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 11:56:17,419 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 11:56:17,420 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:17,420 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:17,420 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 11:56:17,420 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:17,422 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:17,422 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:17,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:17,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:17,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:17,422 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:17,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:17,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:17,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:17,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:17,422 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:17,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:17,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:17,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:17,424 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:17,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:17,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:17,426 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:17,426 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,426 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:17,426 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:17,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:17,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 11:56:17,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:17,426 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:17,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:17,426 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:17,426 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,426 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:17,426 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:17,426 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:17,426 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:17,427 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:17,427 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 11:56:17,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:17,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:17,527 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:17,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:17,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:17,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:17,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:17,531 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:17,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:17,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 11:56:17,535 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 11:56:17,535 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:17,535 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 11:56:17,535 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 11:56:17,535 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:17,535 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:17,536 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:17,536 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:17,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:17,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:17,537 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:17,537 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:17,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:17,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:17,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:17,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 11:56:17,538 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 11:56:17,538 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 11:56:17,538 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 11:56:17,538 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:17,538 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 11:56:17,539 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:17,539 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. started... 2023-03-08 11:56:17,539 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:17,539 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:56:17,539 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:17,539 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff22c265b3d4c9370e800d08d19c30b0 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 11:56:17,539 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. started... 2023-03-08 11:56:17,539 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:17,539 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ffda891a90e48be952988d132e357ad3 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 11:56:17,539 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. started... 2023-03-08 11:56:17,539 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 30ea0c03bb5d1a1afdaa8ef87fc4f610 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 11:56:17,615 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030883a2424bc6f641a49b50615ca12cdfcb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030883a2424bc6f641a49b50615ca12cdfcb 2023-03-08 11:56:17,633 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/1ae4bc06d6e24e2d8a1be316be69232f 2023-03-08 11:56:17,664 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308d563b64ac923455ea8719aa9d387dfd2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308d563b64ac923455ea8719aa9d387dfd2 2023-03-08 11:56:17,669 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/f8fcc6b0b5e940a781ff036f4aa9de95 2023-03-08 11:56:17,672 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/1ae4bc06d6e24e2d8a1be316be69232f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/1ae4bc06d6e24e2d8a1be316be69232f 2023-03-08 11:56:17,697 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/1ae4bc06d6e24e2d8a1be316be69232f, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 11:56:17,698 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/f8fcc6b0b5e940a781ff036f4aa9de95 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/f8fcc6b0b5e940a781ff036f4aa9de95 2023-03-08 11:56:17,703 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/f8fcc6b0b5e940a781ff036f4aa9de95, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 11:56:17,704 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for ff22c265b3d4c9370e800d08d19c30b0 in 165ms, sequenceid=13, compaction requested=true 2023-03-08 11:56:17,704 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:17,704 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:17,704 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:56:18,013 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303087d1356811c0940298cee03ff16e63282 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303087d1356811c0940298cee03ff16e63282 2023-03-08 11:56:18,013 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308e29e44ba96474751b1b2dbec7dca1b95 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308e29e44ba96474751b1b2dbec7dca1b95 2023-03-08 11:56:18,435 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/c5e2321658c14a0894457a939849f02e 2023-03-08 11:56:18,442 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/e12220989ce2414296b6b4b48a234df4 2023-03-08 11:56:18,476 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308289c60a9e9ae4bceaad22fbca11c0161 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308289c60a9e9ae4bceaad22fbca11c0161 2023-03-08 11:56:18,488 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/bbf239aa6ef04d1ca4895eb7744bc08a 2023-03-08 11:56:18,491 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/c5e2321658c14a0894457a939849f02e as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/c5e2321658c14a0894457a939849f02e 2023-03-08 11:56:18,515 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/c5e2321658c14a0894457a939849f02e, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 11:56:18,516 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/bbf239aa6ef04d1ca4895eb7744bc08a as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/bbf239aa6ef04d1ca4895eb7744bc08a 2023-03-08 11:56:18,519 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/bbf239aa6ef04d1ca4895eb7744bc08a, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 11:56:18,520 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 30ea0c03bb5d1a1afdaa8ef87fc4f610 in 980ms, sequenceid=13, compaction requested=true 2023-03-08 11:56:18,520 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:18,520 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:18,520 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:56:18,885 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303085bb7da94ddde49f19987c5ccd57e787a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303085bb7da94ddde49f19987c5ccd57e787a 2023-03-08 11:56:19,129 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 11:56:19,289 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/453babfc06bb417cbea4fa7fd28f1d9d 2023-03-08 11:56:19,294 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/e12220989ce2414296b6b4b48a234df4 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/e12220989ce2414296b6b4b48a234df4 2023-03-08 11:56:19,319 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/e12220989ce2414296b6b4b48a234df4, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 11:56:19,320 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/453babfc06bb417cbea4fa7fd28f1d9d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/453babfc06bb417cbea4fa7fd28f1d9d 2023-03-08 11:56:19,325 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/453babfc06bb417cbea4fa7fd28f1d9d, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 11:56:19,326 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for ffda891a90e48be952988d132e357ad3 in 1787ms, sequenceid=13, compaction requested=true 2023-03-08 11:56:19,326 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:19,326 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:19,326 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:56:19,326 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:56:19,326 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,326 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 11:56:19,326 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 11:56:19,329 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,329 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:19,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:19,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:19,329 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:19,330 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:19,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:19,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:19,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:19,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:19,331 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 11:56:19,331 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e681883[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:19,331 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 11:56:19,331 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:19,332 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:19,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:19,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:19,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 11:56:19,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 11:56:19,332 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:19,332 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,332 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:19,333 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 11:56:19,334 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:19,333 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,334 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 11:56:19,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:19,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:19,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:19,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:19,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:19,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:19,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:19,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:19,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 11:56:19,337 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 11:56:19,337 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:19,337 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:19,337 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 11:56:19,337 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:19,338 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:19,338 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:19,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:19,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:19,338 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:19,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:19,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:19,340 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:19,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:19,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:19,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,341 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:19,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:19,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,350 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,350 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:19,350 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:19,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:19,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:19,351 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 11:56:19,351 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:19,351 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:19,351 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:19,351 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:19,351 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:19,351 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:19,351 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:19,351 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:19,351 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:19,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:19,351 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:19,451 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:19,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:19,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:19,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:19,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 11:56:19,456 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:19,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:19,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 11:56:19,458 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 11:56:19,458 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:19,458 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 11:56:19,458 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 11:56:19,458 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:19,458 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:19,462 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,462 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:19,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:19,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:19,462 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:19,462 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:19,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:19,463 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:19,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:19,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:19,463 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 11:56:19,463 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 11:56:19,463 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 11:56:19,463 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 11:56:19,464 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:19,464 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 11:56:19,464 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:19,464 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:19,464 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:56:19,464 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:19,464 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. started... 2023-03-08 11:56:19,464 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:19,464 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. started... 2023-03-08 11:56:19,464 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. started... 2023-03-08 11:56:19,464 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff22c265b3d4c9370e800d08d19c30b0 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 11:56:19,464 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 30ea0c03bb5d1a1afdaa8ef87fc4f610 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 11:56:19,464 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ffda891a90e48be952988d132e357ad3 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 11:56:19,543 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308c68e7ec3aebc4815896eaf3d9f6b84d3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308c68e7ec3aebc4815896eaf3d9f6b84d3 2023-03-08 11:56:19,549 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308e8b70f45b1524e2ab0a73e5574563fd6 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e8b70f45b1524e2ab0a73e5574563fd6 2023-03-08 11:56:19,569 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/5113220194e548b0bde0414b5ac1f871 2023-03-08 11:56:19,604 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030801925b428ee54cfdaddc7616b8d42d7b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023030801925b428ee54cfdaddc7616b8d42d7b 2023-03-08 11:56:19,609 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/4cb654c094e744e192e3255cb95c80ef 2023-03-08 11:56:19,612 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/5113220194e548b0bde0414b5ac1f871 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/5113220194e548b0bde0414b5ac1f871 2023-03-08 11:56:19,637 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/5113220194e548b0bde0414b5ac1f871, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 11:56:19,638 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/4cb654c094e744e192e3255cb95c80ef as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/4cb654c094e744e192e3255cb95c80ef 2023-03-08 11:56:19,641 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/4cb654c094e744e192e3255cb95c80ef, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 11:56:19,642 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for ff22c265b3d4c9370e800d08d19c30b0 in 178ms, sequenceid=17, compaction requested=true 2023-03-08 11:56:19,642 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:19,642 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:19,642 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:56:19,934 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303089b9fddd18131449b9a8d18f22c25b454 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303089b9fddd18131449b9a8d18f22c25b454 2023-03-08 11:56:19,962 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/1c4dac2c48bd4c40945f5161401f074b 2023-03-08 11:56:20,348 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/91e3e8da83064081b0f4ee3bb2c9a525 2023-03-08 11:56:20,392 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030829ef974fc71d42bd8c119f64f55d244a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023030829ef974fc71d42bd8c119f64f55d244a 2023-03-08 11:56:20,397 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/64d9ed443be044728fa1d25766584b3f 2023-03-08 11:56:20,400 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/1c4dac2c48bd4c40945f5161401f074b as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/1c4dac2c48bd4c40945f5161401f074b 2023-03-08 11:56:20,423 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/1c4dac2c48bd4c40945f5161401f074b, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 11:56:20,423 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/64d9ed443be044728fa1d25766584b3f as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/64d9ed443be044728fa1d25766584b3f 2023-03-08 11:56:20,427 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/64d9ed443be044728fa1d25766584b3f, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 11:56:20,428 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for ffda891a90e48be952988d132e357ad3 in 964ms, sequenceid=17, compaction requested=true 2023-03-08 11:56:20,428 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:20,428 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:20,428 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:56:20,792 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308a7489c35dba641728e3d550841366c78 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308a7489c35dba641728e3d550841366c78 2023-03-08 11:56:21,196 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/465d7fdf219c4b068826ea563cd78940 2023-03-08 11:56:21,200 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/91e3e8da83064081b0f4ee3bb2c9a525 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/91e3e8da83064081b0f4ee3bb2c9a525 2023-03-08 11:56:21,224 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/91e3e8da83064081b0f4ee3bb2c9a525, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 11:56:21,225 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/465d7fdf219c4b068826ea563cd78940 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/465d7fdf219c4b068826ea563cd78940 2023-03-08 11:56:21,229 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/465d7fdf219c4b068826ea563cd78940, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 11:56:21,230 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 30ea0c03bb5d1a1afdaa8ef87fc4f610 in 1766ms, sequenceid=17, compaction requested=true 2023-03-08 11:56:21,230 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:21,231 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:21,231 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:56:21,231 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:56:21,231 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,231 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 11:56:21,231 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 11:56:21,234 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,234 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:21,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:21,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:21,234 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:21,234 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:21,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:21,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:21,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:21,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:21,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 11:56:21,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31d5f38c[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:21,236 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 11:56:21,236 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:21,237 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:21,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:21,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:21,237 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 11:56:21,237 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 11:56:21,237 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:21,237 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,237 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:21,239 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,239 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 11:56:21,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:21,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:21,239 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:21,239 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 11:56:21,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:21,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:21,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:21,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:21,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:21,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:21,241 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 11:56:21,241 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 11:56:21,241 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:21,241 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:21,241 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 11:56:21,241 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:21,243 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:21,243 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:21,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:21,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:21,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:21,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:21,243 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:21,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:21,243 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:21,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:21,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:21,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:21,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:21,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:21,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:21,244 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:21,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:21,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,247 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:21,247 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:21,247 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:21,247 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:21,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:21,247 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:21,247 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 11:56:21,247 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 11:56:21,247 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:21,247 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:21,247 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,247 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:21,247 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:21,247 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:21,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:21,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:21,247 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:21,248 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:21,348 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:21,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:21,357 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:56:21,364 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:56:21,371 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:56:21,372 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:56:21,458 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 11:56:21,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 11:56:21,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 11:56:21,460 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 11:56:21,460 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:21,460 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 11:56:21,460 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 11:56:21,460 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:21,460 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:21,461 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:21,461 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:21,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:21,462 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:21,462 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 11:56:21,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:21,462 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:21,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 11:56:21,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:21,462 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 11:56:21,463 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 11:56:21,463 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 11:56:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 11:56:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 11:56:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 11:56:21,463 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:21,463 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:21,463 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. started... 2023-03-08 11:56:21,463 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. started... 2023-03-08 11:56:21,463 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 11:56:21,463 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ffda891a90e48be952988d132e357ad3 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 11:56:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 11:56:21,463 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff22c265b3d4c9370e800d08d19c30b0 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 11:56:21,463 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. started... 2023-03-08 11:56:21,463 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 30ea0c03bb5d1a1afdaa8ef87fc4f610 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 11:56:21,902 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/0c74b94f582748c2b384c79b72c7de16 2023-03-08 11:56:21,929 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c74b94f582748c2b384c79b72c7de16 2023-03-08 11:56:22,316 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/291c8dc518324798a9ef8801bc655f3e 2023-03-08 11:56:22,316 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/ff5f6b8b0b914098ae413420a30a3625 2023-03-08 11:56:22,340 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/377b961cd254495b8ac0d72c165699d2 2023-03-08 11:56:22,340 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff5f6b8b0b914098ae413420a30a3625 2023-03-08 11:56:22,343 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 377b961cd254495b8ac0d72c165699d2 2023-03-08 11:56:22,344 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/0c74b94f582748c2b384c79b72c7de16 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/0c74b94f582748c2b384c79b72c7de16 2023-03-08 11:56:22,346 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 291c8dc518324798a9ef8801bc655f3e 2023-03-08 11:56:22,361 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/a8d11b5e0f0547529b0da21f55f44f91 2023-03-08 11:56:22,364 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a8d11b5e0f0547529b0da21f55f44f91 2023-03-08 11:56:22,365 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/291c8dc518324798a9ef8801bc655f3e as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/291c8dc518324798a9ef8801bc655f3e 2023-03-08 11:56:22,370 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c74b94f582748c2b384c79b72c7de16 2023-03-08 11:56:22,370 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/0c74b94f582748c2b384c79b72c7de16, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 11:56:22,371 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/377b961cd254495b8ac0d72c165699d2 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/377b961cd254495b8ac0d72c165699d2 2023-03-08 11:56:22,376 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 377b961cd254495b8ac0d72c165699d2 2023-03-08 11:56:22,376 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/377b961cd254495b8ac0d72c165699d2, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 11:56:22,377 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for ffda891a90e48be952988d132e357ad3 in 914ms, sequenceid=23, compaction requested=true 2023-03-08 11:56:22,377 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:22,377 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:22,377 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 11:56:22,392 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 291c8dc518324798a9ef8801bc655f3e 2023-03-08 11:56:22,392 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/291c8dc518324798a9ef8801bc655f3e, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 11:56:22,392 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/a8d11b5e0f0547529b0da21f55f44f91 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/a8d11b5e0f0547529b0da21f55f44f91 2023-03-08 11:56:22,397 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a8d11b5e0f0547529b0da21f55f44f91 2023-03-08 11:56:22,397 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/a8d11b5e0f0547529b0da21f55f44f91, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 11:56:22,398 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 30ea0c03bb5d1a1afdaa8ef87fc4f610 in 935ms, sequenceid=23, compaction requested=true 2023-03-08 11:56:22,398 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:22,398 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:22,398 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 11:56:23,153 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/2adf3e644e0b4b4cb70b8287dcb9b433 2023-03-08 11:56:23,156 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2adf3e644e0b4b4cb70b8287dcb9b433 2023-03-08 11:56:23,157 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/ff5f6b8b0b914098ae413420a30a3625 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/ff5f6b8b0b914098ae413420a30a3625 2023-03-08 11:56:23,179 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff5f6b8b0b914098ae413420a30a3625 2023-03-08 11:56:23,179 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/ff5f6b8b0b914098ae413420a30a3625, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 11:56:23,180 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/2adf3e644e0b4b4cb70b8287dcb9b433 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/2adf3e644e0b4b4cb70b8287dcb9b433 2023-03-08 11:56:23,184 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2adf3e644e0b4b4cb70b8287dcb9b433 2023-03-08 11:56:23,184 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/2adf3e644e0b4b4cb70b8287dcb9b433, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 11:56:23,184 INFO [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for ff22c265b3d4c9370e800d08d19c30b0 in 1721ms, sequenceid=23, compaction requested=true 2023-03-08 11:56:23,184 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:23,184 DEBUG [rs(jenkins-hbase4.apache.org,45551,1678276414946)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:23,184 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 11:56:23,185 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 11:56:23,185 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,185 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 11:56:23,185 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 11:56:23,186 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:23,186 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:23,186 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:23,187 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 11:56:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:23,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45551,1678276414946' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 11:56:23,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35435d90[Count = 0] remaining members to acquire global barrier 2023-03-08 11:56:23,188 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 11:56:23,188 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:23,189 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:23,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:23,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:23,189 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 11:56:23,189 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 11:56:23,189 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45551,1678276414946' in zk 2023-03-08 11:56:23,189 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,189 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 11:56:23,191 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 11:56:23,191 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,191 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:23,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:23,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:23,191 DEBUG [member: 'jenkins-hbase4.apache.org,45551,1678276414946' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 11:56:23,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:23,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:23,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:23,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:23,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:23,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45551,1678276414946': 2023-03-08 11:56:23,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45551,1678276414946' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 11:56:23,194 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 11:56:23,194 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 11:56:23,194 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 11:56:23,194 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 11:56:23,194 INFO [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 11:56:23,196 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:23,196 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:23,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:23,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 11:56:23,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 11:56:23,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:23,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:23,196 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:23,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 11:56:23,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:23,197 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:23,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:23,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:23,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 11:56:23,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:23,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,198 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 11:56:23,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 11:56:23,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,200 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,200 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 11:56:23,200 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:23,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 11:56:23,200 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 11:56:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 11:56:23,200 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 11:56:23,200 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 11:56:23,201 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:23,201 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:23,201 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 11:56:23,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 11:56:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 11:56:23,201 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 11:56:23,201 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 11:56:23,200 DEBUG [(jenkins-hbase4.apache.org,46487,1678276413741)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 11:56:23,201 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 11:56:23,201 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 11:56:23,201 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 11:56:23,301 DEBUG [Listener at localhost/36211] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 11:56:23,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46487] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 11:56:23,303 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:56:23,303 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): ff22c265b3d4c9370e800d08d19c30b0/family1 is initiating major compaction (all files) 2023-03-08 11:56:23,303 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:56:23,303 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:56:23,303 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of ff22c265b3d4c9370e800d08d19c30b0/family1 in testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:23,303 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/e68b4380849049da813135482a908328, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/34b929af3c9841dcb47922b43ecaf397, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/1ae4bc06d6e24e2d8a1be316be69232f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/5113220194e548b0bde0414b5ac1f871, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/ff5f6b8b0b914098ae413420a30a3625] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp, totalSize=27.1 K 2023-03-08 11:56:23,304 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting e68b4380849049da813135482a908328, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276574555 2023-03-08 11:56:23,304 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 34b929af3c9841dcb47922b43ecaf397, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276576029 2023-03-08 11:56:23,304 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 1ae4bc06d6e24e2d8a1be316be69232f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276577529 2023-03-08 11:56:23,305 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 5113220194e548b0bde0414b5ac1f871, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276579454 2023-03-08 11:56:23,305 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting ff5f6b8b0b914098ae413420a30a3625, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:56:23,313 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): ff22c265b3d4c9370e800d08d19c30b0#family1#compaction#196 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:56:23,757 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308539065c7d7fc45e992df7b4825cfc608_del 2023-03-08 11:56:23,757 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308539065c7d7fc45e992df7b4825cfc608_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308539065c7d7fc45e992df7b4825cfc608_del 2023-03-08 11:56:24,614 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e45a2f7230a04f2cbb362b51e3e6f5de 2023-03-08 11:56:24,615 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family1/e45a2f7230a04f2cbb362b51e3e6f5de as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/e45a2f7230a04f2cbb362b51e3e6f5de 2023-03-08 11:56:24,638 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e45a2f7230a04f2cbb362b51e3e6f5de 2023-03-08 11:56:24,639 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ff22c265b3d4c9370e800d08d19c30b0/family1 of ff22c265b3d4c9370e800d08d19c30b0 into e45a2f7230a04f2cbb362b51e3e6f5de(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:56:24,639 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:24,640 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:56:24,640 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): ff22c265b3d4c9370e800d08d19c30b0/family2 is initiating major compaction (all files) 2023-03-08 11:56:24,640 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:56:24,640 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:56:24,640 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of ff22c265b3d4c9370e800d08d19c30b0/family2 in testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:24,640 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/8e40a73f66b440ac9e0a47c3ecf0fac7, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/c25e3152893344dc9dc57618d6325f41, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/f8fcc6b0b5e940a781ff036f4aa9de95, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/4cb654c094e744e192e3255cb95c80ef, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/2adf3e644e0b4b4cb70b8287dcb9b433] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp, totalSize=25.4 K 2023-03-08 11:56:24,640 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 8e40a73f66b440ac9e0a47c3ecf0fac7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276574555 2023-03-08 11:56:24,641 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting c25e3152893344dc9dc57618d6325f41, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276576029 2023-03-08 11:56:24,641 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting f8fcc6b0b5e940a781ff036f4aa9de95, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276577529 2023-03-08 11:56:24,641 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 4cb654c094e744e192e3255cb95c80ef, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276579454 2023-03-08 11:56:24,641 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 2adf3e644e0b4b4cb70b8287dcb9b433, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:56:24,649 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): ff22c265b3d4c9370e800d08d19c30b0#family2#compaction#197 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:56:24,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/e68b4380849049da813135482a908328, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/34b929af3c9841dcb47922b43ecaf397, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/1ae4bc06d6e24e2d8a1be316be69232f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/5113220194e548b0bde0414b5ac1f871, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/ff5f6b8b0b914098ae413420a30a3625] to archive 2023-03-08 11:56:24,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:24,675 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/e68b4380849049da813135482a908328 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/e68b4380849049da813135482a908328 2023-03-08 11:56:24,676 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/34b929af3c9841dcb47922b43ecaf397 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/34b929af3c9841dcb47922b43ecaf397 2023-03-08 11:56:24,677 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/1ae4bc06d6e24e2d8a1be316be69232f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/1ae4bc06d6e24e2d8a1be316be69232f 2023-03-08 11:56:24,678 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/5113220194e548b0bde0414b5ac1f871 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/5113220194e548b0bde0414b5ac1f871 2023-03-08 11:56:24,679 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/ff5f6b8b0b914098ae413420a30a3625 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/ff5f6b8b0b914098ae413420a30a3625 2023-03-08 11:56:25,057 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308ac3f8c92d1ba496ba3f446db9308aea6_del 2023-03-08 11:56:25,058 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308ac3f8c92d1ba496ba3f446db9308aea6_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308ac3f8c92d1ba496ba3f446db9308aea6_del 2023-03-08 11:56:25,869 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5464e1f116c9492bae802da5c48e4c43 2023-03-08 11:56:25,870 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/.tmp/family2/5464e1f116c9492bae802da5c48e4c43 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/5464e1f116c9492bae802da5c48e4c43 2023-03-08 11:56:25,872 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5464e1f116c9492bae802da5c48e4c43 2023-03-08 11:56:25,873 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ff22c265b3d4c9370e800d08d19c30b0/family2 of ff22c265b3d4c9370e800d08d19c30b0 into 5464e1f116c9492bae802da5c48e4c43(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:56:25,873 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:25,873 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:56:25,873 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): ffda891a90e48be952988d132e357ad3/family1 is initiating major compaction (all files) 2023-03-08 11:56:25,874 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:56:25,874 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:56:25,874 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of ffda891a90e48be952988d132e357ad3/family1 in testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:25,874 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/bf9c9528256f4e72a4bab498d62a87b7, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/ef424c2cff2f4557b30bf138d3d4f8cf, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/e12220989ce2414296b6b4b48a234df4, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/1c4dac2c48bd4c40945f5161401f074b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/0c74b94f582748c2b384c79b72c7de16] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp, totalSize=27.1 K 2023-03-08 11:56:25,874 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting bf9c9528256f4e72a4bab498d62a87b7, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276574554 2023-03-08 11:56:25,875 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting ef424c2cff2f4557b30bf138d3d4f8cf, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276576029 2023-03-08 11:56:25,875 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting e12220989ce2414296b6b4b48a234df4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276577529 2023-03-08 11:56:25,875 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 1c4dac2c48bd4c40945f5161401f074b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276579453 2023-03-08 11:56:25,875 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 0c74b94f582748c2b384c79b72c7de16, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:56:25,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/8e40a73f66b440ac9e0a47c3ecf0fac7, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/c25e3152893344dc9dc57618d6325f41, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/f8fcc6b0b5e940a781ff036f4aa9de95, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/4cb654c094e744e192e3255cb95c80ef, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/2adf3e644e0b4b4cb70b8287dcb9b433] to archive 2023-03-08 11:56:25,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:25,882 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): ffda891a90e48be952988d132e357ad3#family1#compaction#198 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:56:25,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/8e40a73f66b440ac9e0a47c3ecf0fac7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/8e40a73f66b440ac9e0a47c3ecf0fac7 2023-03-08 11:56:25,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/c25e3152893344dc9dc57618d6325f41 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/c25e3152893344dc9dc57618d6325f41 2023-03-08 11:56:25,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/f8fcc6b0b5e940a781ff036f4aa9de95 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/f8fcc6b0b5e940a781ff036f4aa9de95 2023-03-08 11:56:25,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/4cb654c094e744e192e3255cb95c80ef to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/4cb654c094e744e192e3255cb95c80ef 2023-03-08 11:56:25,886 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/2adf3e644e0b4b4cb70b8287dcb9b433 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family2/2adf3e644e0b4b4cb70b8287dcb9b433 2023-03-08 11:56:26,319 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308a70c90acf86344a3bbe408bd568ff76e_del 2023-03-08 11:56:26,320 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308a70c90acf86344a3bbe408bd568ff76e_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308a70c90acf86344a3bbe408bd568ff76e_del 2023-03-08 11:56:27,176 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3e98a231e3b84b7798be9191e59e9442 2023-03-08 11:56:27,177 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family1/3e98a231e3b84b7798be9191e59e9442 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/3e98a231e3b84b7798be9191e59e9442 2023-03-08 11:56:27,198 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3e98a231e3b84b7798be9191e59e9442 2023-03-08 11:56:27,199 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ffda891a90e48be952988d132e357ad3/family1 of ffda891a90e48be952988d132e357ad3 into 3e98a231e3b84b7798be9191e59e9442(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:56:27,199 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:27,199 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:56:27,199 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): ffda891a90e48be952988d132e357ad3/family2 is initiating major compaction (all files) 2023-03-08 11:56:27,200 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:56:27,200 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:56:27,201 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of ffda891a90e48be952988d132e357ad3/family2 in testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:27,201 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/7e58d655a9674f788d12c3fbcf88d537, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/cc7faf7b68144d0aafd1b2941f93f97f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/453babfc06bb417cbea4fa7fd28f1d9d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/64d9ed443be044728fa1d25766584b3f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/377b961cd254495b8ac0d72c165699d2] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp, totalSize=25.4 K 2023-03-08 11:56:27,202 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 7e58d655a9674f788d12c3fbcf88d537, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276574554 2023-03-08 11:56:27,202 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting cc7faf7b68144d0aafd1b2941f93f97f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276576029 2023-03-08 11:56:27,203 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 453babfc06bb417cbea4fa7fd28f1d9d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276577529 2023-03-08 11:56:27,203 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 64d9ed443be044728fa1d25766584b3f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276579453 2023-03-08 11:56:27,203 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 377b961cd254495b8ac0d72c165699d2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:56:27,210 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): ffda891a90e48be952988d132e357ad3#family2#compaction#199 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:56:27,273 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/bf9c9528256f4e72a4bab498d62a87b7, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/ef424c2cff2f4557b30bf138d3d4f8cf, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/e12220989ce2414296b6b4b48a234df4, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/1c4dac2c48bd4c40945f5161401f074b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/0c74b94f582748c2b384c79b72c7de16] to archive 2023-03-08 11:56:27,274 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:27,275 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/bf9c9528256f4e72a4bab498d62a87b7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/bf9c9528256f4e72a4bab498d62a87b7 2023-03-08 11:56:27,276 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/ef424c2cff2f4557b30bf138d3d4f8cf to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/ef424c2cff2f4557b30bf138d3d4f8cf 2023-03-08 11:56:27,277 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/e12220989ce2414296b6b4b48a234df4 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/e12220989ce2414296b6b4b48a234df4 2023-03-08 11:56:27,278 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/1c4dac2c48bd4c40945f5161401f074b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/1c4dac2c48bd4c40945f5161401f074b 2023-03-08 11:56:27,279 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/0c74b94f582748c2b384c79b72c7de16 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/0c74b94f582748c2b384c79b72c7de16 2023-03-08 11:56:27,617 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303082fb68d44ee6b4faaa00673a1045a90d5_del 2023-03-08 11:56:27,617 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303082fb68d44ee6b4faaa00673a1045a90d5_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303082fb68d44ee6b4faaa00673a1045a90d5_del 2023-03-08 11:56:28,430 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ae50753791644b4bbb7ac84a2aa220a 2023-03-08 11:56:28,430 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/.tmp/family2/2ae50753791644b4bbb7ac84a2aa220a as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/2ae50753791644b4bbb7ac84a2aa220a 2023-03-08 11:56:28,433 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ae50753791644b4bbb7ac84a2aa220a 2023-03-08 11:56:28,434 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ffda891a90e48be952988d132e357ad3/family2 of ffda891a90e48be952988d132e357ad3 into 2ae50753791644b4bbb7ac84a2aa220a(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:56:28,434 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:28,434 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:56:28,434 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): 30ea0c03bb5d1a1afdaa8ef87fc4f610/family1 is initiating major compaction (all files) 2023-03-08 11:56:28,434 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:56:28,435 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:56:28,435 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of 30ea0c03bb5d1a1afdaa8ef87fc4f610/family1 in testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:28,435 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/e3dbcae5d745483f8af5f980d25e26f1, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/55fda5e064f24c9390fa240e6941174f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/c5e2321658c14a0894457a939849f02e, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/91e3e8da83064081b0f4ee3bb2c9a525, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/291c8dc518324798a9ef8801bc655f3e] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp, totalSize=27.1 K 2023-03-08 11:56:28,435 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting e3dbcae5d745483f8af5f980d25e26f1, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276574555 2023-03-08 11:56:28,435 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 55fda5e064f24c9390fa240e6941174f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276576029 2023-03-08 11:56:28,436 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting c5e2321658c14a0894457a939849f02e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276577529 2023-03-08 11:56:28,436 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 91e3e8da83064081b0f4ee3bb2c9a525, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276579454 2023-03-08 11:56:28,436 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 291c8dc518324798a9ef8801bc655f3e, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:56:28,443 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): 30ea0c03bb5d1a1afdaa8ef87fc4f610#family1#compaction#200 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:56:28,472 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/7e58d655a9674f788d12c3fbcf88d537, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/cc7faf7b68144d0aafd1b2941f93f97f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/453babfc06bb417cbea4fa7fd28f1d9d, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/64d9ed443be044728fa1d25766584b3f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/377b961cd254495b8ac0d72c165699d2] to archive 2023-03-08 11:56:28,473 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:28,474 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/7e58d655a9674f788d12c3fbcf88d537 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/7e58d655a9674f788d12c3fbcf88d537 2023-03-08 11:56:28,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/cc7faf7b68144d0aafd1b2941f93f97f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/cc7faf7b68144d0aafd1b2941f93f97f 2023-03-08 11:56:28,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/453babfc06bb417cbea4fa7fd28f1d9d to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/453babfc06bb417cbea4fa7fd28f1d9d 2023-03-08 11:56:28,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/64d9ed443be044728fa1d25766584b3f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/64d9ed443be044728fa1d25766584b3f 2023-03-08 11:56:28,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/377b961cd254495b8ac0d72c165699d2 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family2/377b961cd254495b8ac0d72c165699d2 2023-03-08 11:56:28,886 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:56:28,887 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x41dcee86 to 127.0.0.1:51863 2023-03-08 11:56:28,887 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:56:28,893 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308c64a9cf40a4d476f8431cc806d0206ce_del 2023-03-08 11:56:28,893 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308c64a9cf40a4d476f8431cc806d0206ce_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308c64a9cf40a4d476f8431cc806d0206ce_del 2023-03-08 11:56:29,745 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bbecb305ed8f4be182b75500f4a55e51 2023-03-08 11:56:29,746 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family1/bbecb305ed8f4be182b75500f4a55e51 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/bbecb305ed8f4be182b75500f4a55e51 2023-03-08 11:56:29,767 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bbecb305ed8f4be182b75500f4a55e51 2023-03-08 11:56:29,768 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 30ea0c03bb5d1a1afdaa8ef87fc4f610/family1 of 30ea0c03bb5d1a1afdaa8ef87fc4f610 into bbecb305ed8f4be182b75500f4a55e51(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:56:29,768 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:29,768 DEBUG [Listener at localhost/36211] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 11:56:29,768 DEBUG [Listener at localhost/36211] regionserver.HStore(1912): 30ea0c03bb5d1a1afdaa8ef87fc4f610/family2 is initiating major compaction (all files) 2023-03-08 11:56:29,768 INFO [Listener at localhost/36211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 11:56:29,769 INFO [Listener at localhost/36211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 11:56:29,769 INFO [Listener at localhost/36211] regionserver.HRegion(2259): Starting compaction of 30ea0c03bb5d1a1afdaa8ef87fc4f610/family2 in testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:29,769 INFO [Listener at localhost/36211] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/599c0c789b0448469e0773c0801a733b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/d18e572dcf704bcca507adb57c00b124, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/bbf239aa6ef04d1ca4895eb7744bc08a, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/465d7fdf219c4b068826ea563cd78940, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/a8d11b5e0f0547529b0da21f55f44f91] into tmpdir=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp, totalSize=25.4 K 2023-03-08 11:56:29,769 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 599c0c789b0448469e0773c0801a733b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678276574555 2023-03-08 11:56:29,769 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting d18e572dcf704bcca507adb57c00b124, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678276576029 2023-03-08 11:56:29,770 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting bbf239aa6ef04d1ca4895eb7744bc08a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678276577529 2023-03-08 11:56:29,770 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting 465d7fdf219c4b068826ea563cd78940, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678276579454 2023-03-08 11:56:29,771 DEBUG [Listener at localhost/36211] compactions.Compactor(207): Compacting a8d11b5e0f0547529b0da21f55f44f91, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 11:56:29,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/e3dbcae5d745483f8af5f980d25e26f1, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/55fda5e064f24c9390fa240e6941174f, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/c5e2321658c14a0894457a939849f02e, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/91e3e8da83064081b0f4ee3bb2c9a525, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/291c8dc518324798a9ef8801bc655f3e] to archive 2023-03-08 11:56:29,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:29,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/e3dbcae5d745483f8af5f980d25e26f1 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/e3dbcae5d745483f8af5f980d25e26f1 2023-03-08 11:56:29,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/55fda5e064f24c9390fa240e6941174f to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/55fda5e064f24c9390fa240e6941174f 2023-03-08 11:56:29,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/c5e2321658c14a0894457a939849f02e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/c5e2321658c14a0894457a939849f02e 2023-03-08 11:56:29,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/91e3e8da83064081b0f4ee3bb2c9a525 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/91e3e8da83064081b0f4ee3bb2c9a525 2023-03-08 11:56:29,778 INFO [Listener at localhost/36211] throttle.PressureAwareThroughputController(145): 30ea0c03bb5d1a1afdaa8ef87fc4f610#family2#compaction#201 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 11:56:29,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/291c8dc518324798a9ef8801bc655f3e to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/291c8dc518324798a9ef8801bc655f3e 2023-03-08 11:56:30,187 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303080bf4f46b380a472b89990c0e11c0e8f9_del 2023-03-08 11:56:30,188 INFO [Listener at localhost/36211] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303080bf4f46b380a472b89990c0e11c0e8f9_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303080bf4f46b380a472b89990c0e11c0e8f9_del 2023-03-08 11:56:31,000 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb2f3d52e7134692aba245cb05c1de8d 2023-03-08 11:56:31,000 DEBUG [Listener at localhost/36211] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/.tmp/family2/eb2f3d52e7134692aba245cb05c1de8d as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/eb2f3d52e7134692aba245cb05c1de8d 2023-03-08 11:56:31,003 INFO [Listener at localhost/36211] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb2f3d52e7134692aba245cb05c1de8d 2023-03-08 11:56:31,004 INFO [Listener at localhost/36211] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 30ea0c03bb5d1a1afdaa8ef87fc4f610/family2 of 30ea0c03bb5d1a1afdaa8ef87fc4f610 into eb2f3d52e7134692aba245cb05c1de8d(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 11:56:31,004 DEBUG [Listener at localhost/36211] regionserver.HRegion(2289): Compaction status journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:31,007 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:56:31,009 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:56:31,012 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:56:31,013 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:56:31,017 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46487] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10r\xB6\xD9v\xAB\xE3\xBE\x0C_\x94+\xDBA\x02\x02G"\x10\xB3\xB5\xEB:"2\xE3k\x88\xF9\x16G'*,\x15*\x10\xC4\x82a\xC2d\xE2\xB7\xA3\x8DL\x12\xBE&\xE7xG2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-08 11:56:31,038 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 11:56:31,039 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 11:56:31,039 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:56:31,039 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678277191039 2023-03-08 11:56:31,040 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 11:56:31,074 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/599c0c789b0448469e0773c0801a733b, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/d18e572dcf704bcca507adb57c00b124, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/bbf239aa6ef04d1ca4895eb7744bc08a, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/465d7fdf219c4b068826ea563cd78940, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/a8d11b5e0f0547529b0da21f55f44f91] to archive 2023-03-08 11:56:31,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:31,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/599c0c789b0448469e0773c0801a733b to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/599c0c789b0448469e0773c0801a733b 2023-03-08 11:56:31,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/d18e572dcf704bcca507adb57c00b124 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/d18e572dcf704bcca507adb57c00b124 2023-03-08 11:56:31,082 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/bbf239aa6ef04d1ca4895eb7744bc08a to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/bbf239aa6ef04d1ca4895eb7744bc08a 2023-03-08 11:56:31,083 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/465d7fdf219c4b068826ea563cd78940 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/465d7fdf219c4b068826ea563cd78940 2023-03-08 11:56:31,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/a8d11b5e0f0547529b0da21f55f44f91 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family2/a8d11b5e0f0547529b0da21f55f44f91 2023-03-08 11:56:31,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-08 11:56:31,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308a70c90acf86344a3bbe408bd568ff76e_del 2023-03-08 11:56:31,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308539065c7d7fc45e992df7b4825cfc608_del 2023-03-08 11:56:31,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308c64a9cf40a4d476f8431cc806d0206ce_del 2023-03-08 11:56:31,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 11:56:31,429 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] zookeeper.ReadOnlyZKClient(139): Connect 0x4f465614 to 127.0.0.1:51863 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 11:56:31,433 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a2a7160, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 11:56:31,434 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 11:56:31,599 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 11:56:32,374 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308fe00478c723d4e4f914584ba706f1aa7 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308fe00478c723d4e4f914584ba706f1aa7 2023-03-08 11:56:32,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 11:56:32,385 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 11:56:32,387 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53916, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 11:56:32,395 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 11:56:32,396 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36840, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 11:56:32,422 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/cbb8950973124ac6ae3aba7cac01ebc8 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-08 11:56:32,423 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/cbb8950973124ac6ae3aba7cac01ebc8}] 2023-03-08 11:56:32,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/cbb8950973124ac6ae3aba7cac01ebc8 for inclusion in 30ea0c03bb5d1a1afdaa8ef87fc4f610/family1 2023-03-08 11:56:32,445 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 11:56:32,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-08 11:56:32,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 11:56:32,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:32,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/cbb8950973124ac6ae3aba7cac01ebc8 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__npda5b4jdt94ekmr8jqtdajjl1h6rgnc4v59va1eehjt4g9j14h3qqpo3vqoaugm/family1/cbb8950973124ac6ae3aba7cac01ebc8 2023-03-08 11:56:32,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__npda5b4jdt94ekmr8jqtdajjl1h6rgnc4v59va1eehjt4g9j14h3qqpo3vqoaugm/family1/cbb8950973124ac6ae3aba7cac01ebc8 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/2d744456370d4e18b628f9aafd88008a_SeqId_28_ 2023-03-08 11:56:32,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__npda5b4jdt94ekmr8jqtdajjl1h6rgnc4v59va1eehjt4g9j14h3qqpo3vqoaugm/family1/cbb8950973124ac6ae3aba7cac01ebc8 into 30ea0c03bb5d1a1afdaa8ef87fc4f610/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/2d744456370d4e18b628f9aafd88008a_SeqId_28_ - updating store file list. 2023-03-08 11:56:32,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/2d744456370d4e18b628f9aafd88008a_SeqId_28_ into 30ea0c03bb5d1a1afdaa8ef87fc4f610/family1 2023-03-08 11:56:32,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__npda5b4jdt94ekmr8jqtdajjl1h6rgnc4v59va1eehjt4g9j14h3qqpo3vqoaugm/family1/cbb8950973124ac6ae3aba7cac01ebc8 into 30ea0c03bb5d1a1afdaa8ef87fc4f610/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/family1/2d744456370d4e18b628f9aafd88008a_SeqId_28_) 2023-03-08 11:56:32,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__npda5b4jdt94ekmr8jqtdajjl1h6rgnc4v59va1eehjt4g9j14h3qqpo3vqoaugm/family1/cbb8950973124ac6ae3aba7cac01ebc8 2023-03-08 11:56:32,480 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 11:56:32,481 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:32,482 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303087a0b85431a3e4749ba6a7d7aaafa8060 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303087a0b85431a3e4749ba6a7d7aaafa8060 2023-03-08 11:56:32,483 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030884b3806a33f8476c82663e06ef7005fb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030884b3806a33f8476c82663e06ef7005fb 2023-03-08 11:56:32,483 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303089b9fddd18131449b9a8d18f22c25b454 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303089b9fddd18131449b9a8d18f22c25b454 2023-03-08 11:56:32,484 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308e29e44ba96474751b1b2dbec7dca1b95 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308e29e44ba96474751b1b2dbec7dca1b95 2023-03-08 11:56:32,484 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 11:56:32,484 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 11:56:32,485 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 11:56:32,485 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 11:56:33,033 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303087a9a55dcb7e846728b782cba3836b7d6 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303087a9a55dcb7e846728b782cba3836b7d6 2023-03-08 11:56:33,034 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 11:56:33,072 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/006c6e0474c04928b66dbd5b26529c49 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-08 11:56:33,073 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/006c6e0474c04928b66dbd5b26529c49}] 2023-03-08 11:56:33,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/006c6e0474c04928b66dbd5b26529c49 for inclusion in ffda891a90e48be952988d132e357ad3/family1 2023-03-08 11:56:33,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-08 11:56:33,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 11:56:33,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:33,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/006c6e0474c04928b66dbd5b26529c49 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__3tecotp8m6tpb5q65vaa7fk2clcgedh8tpt9ja2a9h1moe25qfdt17ic8fckndqb/family1/006c6e0474c04928b66dbd5b26529c49 2023-03-08 11:56:33,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__3tecotp8m6tpb5q65vaa7fk2clcgedh8tpt9ja2a9h1moe25qfdt17ic8fckndqb/family1/006c6e0474c04928b66dbd5b26529c49 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/b8bdef6f45f944e3ac57337bf76d5354_SeqId_28_ 2023-03-08 11:56:33,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__3tecotp8m6tpb5q65vaa7fk2clcgedh8tpt9ja2a9h1moe25qfdt17ic8fckndqb/family1/006c6e0474c04928b66dbd5b26529c49 into ffda891a90e48be952988d132e357ad3/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/b8bdef6f45f944e3ac57337bf76d5354_SeqId_28_ - updating store file list. 2023-03-08 11:56:33,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/b8bdef6f45f944e3ac57337bf76d5354_SeqId_28_ into ffda891a90e48be952988d132e357ad3/family1 2023-03-08 11:56:33,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__3tecotp8m6tpb5q65vaa7fk2clcgedh8tpt9ja2a9h1moe25qfdt17ic8fckndqb/family1/006c6e0474c04928b66dbd5b26529c49 into ffda891a90e48be952988d132e357ad3/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/family1/b8bdef6f45f944e3ac57337bf76d5354_SeqId_28_) 2023-03-08 11:56:33,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__3tecotp8m6tpb5q65vaa7fk2clcgedh8tpt9ja2a9h1moe25qfdt17ic8fckndqb/family1/006c6e0474c04928b66dbd5b26529c49 2023-03-08 11:56:33,119 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 11:56:33,119 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:33,120 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303087d1356811c0940298cee03ff16e63282 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303087d1356811c0940298cee03ff16e63282 2023-03-08 11:56:33,121 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308857bbc733d0d46ada2c1d363512d216c to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308857bbc733d0d46ada2c1d363512d216c 2023-03-08 11:56:33,122 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308c68e7ec3aebc4815896eaf3d9f6b84d3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308c68e7ec3aebc4815896eaf3d9f6b84d3 2023-03-08 11:56:33,122 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308ce30166840a143b6897b6d2ec4647016 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308ce30166840a143b6897b6d2ec4647016 2023-03-08 11:56:33,123 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 11:56:33,123 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 11:56:33,123 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 11:56:33,123 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 11:56:34,069 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308b7fbbe2dc5db4ede953875a62cef3a19 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308b7fbbe2dc5db4ede953875a62cef3a19 2023-03-08 11:56:34,070 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 11:56:34,112 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/5867fcbcab3f4bc1a64db110f11fe293 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-08 11:56:34,113 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0., hostname=jenkins-hbase4.apache.org,45551,1678276414946, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/5867fcbcab3f4bc1a64db110f11fe293}] 2023-03-08 11:56:34,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(763): Validating hfile at hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/5867fcbcab3f4bc1a64db110f11fe293 for inclusion in ff22c265b3d4c9370e800d08d19c30b0/family1 2023-03-08 11:56:34,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-08 11:56:34,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 11:56:34,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegion(2446): Flush status journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:34,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/5867fcbcab3f4bc1a64db110f11fe293 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__g767gq1d14idl5v73ia6nvee2vktm64f614e5rhhocl6ql2tohq8l16refclvjs7/family1/5867fcbcab3f4bc1a64db110f11fe293 2023-03-08 11:56:34,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__g767gq1d14idl5v73ia6nvee2vktm64f614e5rhhocl6ql2tohq8l16refclvjs7/family1/5867fcbcab3f4bc1a64db110f11fe293 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/1bc78209c0cf4bf3999ee3343fe2cf2e_SeqId_28_ 2023-03-08 11:56:34,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(849): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__g767gq1d14idl5v73ia6nvee2vktm64f614e5rhhocl6ql2tohq8l16refclvjs7/family1/5867fcbcab3f4bc1a64db110f11fe293 into ff22c265b3d4c9370e800d08d19c30b0/family1 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/1bc78209c0cf4bf3999ee3343fe2cf2e_SeqId_28_ - updating store file list. 2023-03-08 11:56:34,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(882): Loaded HFile hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/1bc78209c0cf4bf3999ee3343fe2cf2e_SeqId_28_ into ff22c265b3d4c9370e800d08d19c30b0/family1 2023-03-08 11:56:34,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.HStore(855): Successfully loaded hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__g767gq1d14idl5v73ia6nvee2vktm64f614e5rhhocl6ql2tohq8l16refclvjs7/family1/5867fcbcab3f4bc1a64db110f11fe293 into ff22c265b3d4c9370e800d08d19c30b0/family1 (new location: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/family1/1bc78209c0cf4bf3999ee3343fe2cf2e_SeqId_28_) 2023-03-08 11:56:34,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45551] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/staging/jenkins__testMajorCompactionFromAdmin__g767gq1d14idl5v73ia6nvee2vktm64f614e5rhhocl6ql2tohq8l16refclvjs7/family1/5867fcbcab3f4bc1a64db110f11fe293 2023-03-08 11:56:34,174 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 11:56:34,175 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:34,176 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030883a2424bc6f641a49b50615ca12cdfcb to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030883a2424bc6f641a49b50615ca12cdfcb 2023-03-08 11:56:34,177 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030890ae4eb634d744898a66004cc40c17f3 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030890ae4eb634d744898a66004cc40c17f3 2023-03-08 11:56:34,178 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308b5a29edef35a44d699b9207d1c581692 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308b5a29edef35a44d699b9207d1c581692 2023-03-08 11:56:34,179 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e8b70f45b1524e2ab0a73e5574563fd6 to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e8b70f45b1524e2ab0a73e5574563fd6 2023-03-08 11:56:34,179 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 11:56:34,179 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 11:56:34,179 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 11:56:34,179 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 11:56:34,179 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 11:56:34,179 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308a70c90acf86344a3bbe408bd568ff76e_del] 2023-03-08 11:56:34,180 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:34,181 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308a70c90acf86344a3bbe408bd568ff76e_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308a70c90acf86344a3bbe408bd568ff76e_del 2023-03-08 11:56:34,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 11:56:34,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308539065c7d7fc45e992df7b4825cfc608_del] 2023-03-08 11:56:34,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:34,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308539065c7d7fc45e992df7b4825cfc608_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308539065c7d7fc45e992df7b4825cfc608_del 2023-03-08 11:56:34,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 11:56:34,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308c64a9cf40a4d476f8431cc806d0206ce_del] 2023-03-08 11:56:34,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 11:56:34,184 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308c64a9cf40a4d476f8431cc806d0206ce_del to hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308c64a9cf40a4d476f8431cc806d0206ce_del 2023-03-08 11:56:34,184 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 11:56:34,184 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678276418409] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 11:56:34,185 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 11:56:34,187 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.1480 sec 2023-03-08 11:56:34,203 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:56:34,204 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:56:34,206 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 11:56:34,207 DEBUG [Listener at localhost/36211] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 11:56:34,210 INFO [Listener at localhost/36211] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 11:56:34,244 INFO [Listener at localhost/36211] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=246 (was 255), OpenFileDescriptor=618 (was 642), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=158 (was 155) - SystemLoadAverage LEAK? -, ProcessCount=173 (was 173), AvailableMemoryMB=7197 (was 7342) 2023-03-08 11:56:34,245 DEBUG [Listener at localhost/36211] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x47652242 to 127.0.0.1:51863 2023-03-08 11:56:34,245 DEBUG [Listener at localhost/36211] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:56:34,245 INFO [Listener at localhost/36211] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-08 11:56:34,246 INFO [Listener at localhost/36211] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 11:56:34,246 DEBUG [Listener at localhost/36211] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5759ba4d to 127.0.0.1:51863 2023-03-08 11:56:34,246 DEBUG [Listener at localhost/36211] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:56:34,246 DEBUG [Listener at localhost/36211] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-08 11:56:34,246 DEBUG [Listener at localhost/36211] util.JVMClusterUtil(257): Found active master hash=1984437194, stopped=false 2023-03-08 11:56:34,246 INFO [Listener at localhost/36211] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,46487,1678276413741 2023-03-08 11:56:34,248 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-08 11:56:34,248 INFO [Listener at localhost/36211] procedure2.ProcedureExecutor(629): Stopping 2023-03-08 11:56:34,248 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-08 11:56:34,248 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 11:56:34,248 DEBUG [Listener at localhost/36211] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x62758e02 to 127.0.0.1:51863 2023-03-08 11:56:34,248 DEBUG [Listener at localhost/36211] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:56:34,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 11:56:34,248 INFO [Listener at localhost/36211] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,45551,1678276414946' ***** 2023-03-08 11:56:34,248 INFO [Listener at localhost/36211] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-08 11:56:34,249 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 11:56:34,249 INFO [RS:0;jenkins-hbase4:45551] regionserver.HeapMemoryManager(220): Stopping 2023-03-08 11:56:34,249 INFO [RS:0;jenkins-hbase4:45551] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-08 11:56:34,249 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-08 11:56:34,249 INFO [RS:0;jenkins-hbase4:45551] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-08 11:56:34,249 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 2821201dfecad47e22ed1f74a6c77989 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 64864abb7d0098b010ede90e39027ce9 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for c154d984bcb433ac75837c927dc4662b 2023-03-08 11:56:34,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2821201dfecad47e22ed1f74a6c77989, disabling compactions & flushes 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 8bbbd6b703fb6a51c3770af58f047d50 2023-03-08 11:56:34,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 09169a8d7fcfaa680f26277a4640bff1 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 148f994b12b7c170e5fbb2fd36359918 2023-03-08 11:56:34,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for e53083d9a161a7082c38a975fc8a0192 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 1b2090f00e0a2d838e28358e6d6297e4 2023-03-08 11:56:34,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. after waiting 0 ms 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for b617fde530c0c1bebf3fa2fa6eab77bb 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 8fd6f903c75ee810ecad794fff5abb6e 2023-03-08 11:56:34,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for ff22c265b3d4c9370e800d08d19c30b0 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 836c4c9f42bf977b9123d7bbe0300d0d 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 91d8ee5ab3b818ebe215b0c4873b5950 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for bb0058b62005e791ee17613797cadc96 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:34,250 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:56:34,251 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 6249f9a64ffef0dbc4b217b148070179 2023-03-08 11:56:34,251 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 6334f697e0e777acd41789a8c25bc8c9 2023-03-08 11:56:34,251 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 8a2814671292e9b2760956bf02135606 2023-03-08 11:56:34,251 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 30ea0c03bb5d1a1afdaa8ef87fc4f610 2023-03-08 11:56:34,251 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:34,251 DEBUG [RS:0;jenkins-hbase4:45551] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x59cbeeba to 127.0.0.1:51863 2023-03-08 11:56:34,251 DEBUG [RS:0;jenkins-hbase4:45551] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:56:34,251 INFO [RS:0;jenkins-hbase4:45551] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-08 11:56:34,251 INFO [RS:0;jenkins-hbase4:45551] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-08 11:56:34,251 INFO [RS:0;jenkins-hbase4:45551] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-08 11:56:34,251 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-08 11:56:34,251 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-08 11:56:34,252 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-08 11:56:34,252 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-08 11:56:34,252 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-08 11:56:34,252 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-08 11:56:34,252 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-08 11:56:34,252 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-08 11:56:34,252 DEBUG [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1505): Waiting on 09169a8d7fcfaa680f26277a4640bff1, 148f994b12b7c170e5fbb2fd36359918, 1588230740, 1b2090f00e0a2d838e28358e6d6297e4, 2821201dfecad47e22ed1f74a6c77989, 30ea0c03bb5d1a1afdaa8ef87fc4f610, 6249f9a64ffef0dbc4b217b148070179, 6334f697e0e777acd41789a8c25bc8c9, 64864abb7d0098b010ede90e39027ce9, 836c4c9f42bf977b9123d7bbe0300d0d, 8a2814671292e9b2760956bf02135606, 8bbbd6b703fb6a51c3770af58f047d50, 8fd6f903c75ee810ecad794fff5abb6e, 91d8ee5ab3b818ebe215b0c4873b5950, b617fde530c0c1bebf3fa2fa6eab77bb, bb0058b62005e791ee17613797cadc96, c154d984bcb433ac75837c927dc4662b, e53083d9a161a7082c38a975fc8a0192, f1ce88c61365d5c7b6e6a24b8a887540, ff22c265b3d4c9370e800d08d19c30b0, ffda891a90e48be952988d132e357ad3 2023-03-08 11:56:34,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2821201dfecad47e22ed1f74a6c77989/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-08 11:56:34,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:56:34,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2821201dfecad47e22ed1f74a6c77989: 2023-03-08 11:56:34,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678276490804.2821201dfecad47e22ed1f74a6c77989. 2023-03-08 11:56:34,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 64864abb7d0098b010ede90e39027ce9, disabling compactions & flushes 2023-03-08 11:56:34,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:56:34,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:56:34,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. after waiting 0 ms 2023-03-08 11:56:34,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:56:34,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/64864abb7d0098b010ede90e39027ce9/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 11:56:34,263 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/.tmp/info/c01256d098904c3c8d95bb504d713fef 2023-03-08 11:56:34,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:56:34,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 64864abb7d0098b010ede90e39027ce9: 2023-03-08 11:56:34,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1678276539566.64864abb7d0098b010ede90e39027ce9. 2023-03-08 11:56:34,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c154d984bcb433ac75837c927dc4662b, disabling compactions & flushes 2023-03-08 11:56:34,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:56:34,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:56:34,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. after waiting 0 ms 2023-03-08 11:56:34,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:56:34,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithWeeklyPolicy/c154d984bcb433ac75837c927dc4662b/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-08 11:56:34,267 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c01256d098904c3c8d95bb504d713fef 2023-03-08 11:56:34,268 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:56:34,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c154d984bcb433ac75837c927dc4662b: 2023-03-08 11:56:34,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1678276445829.c154d984bcb433ac75837c927dc4662b. 2023-03-08 11:56:34,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8bbbd6b703fb6a51c3770af58f047d50, disabling compactions & flushes 2023-03-08 11:56:34,268 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:56:34,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:56:34,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. after waiting 0 ms 2023-03-08 11:56:34,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:56:34,270 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-03-08 11:56:34,270 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-03-08 11:56:34,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8bbbd6b703fb6a51c3770af58f047d50/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-08 11:56:34,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:56:34,273 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-08 11:56:34,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8bbbd6b703fb6a51c3770af58f047d50: 2023-03-08 11:56:34,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1678276457225.8bbbd6b703fb6a51c3770af58f047d50. 2023-03-08 11:56:34,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 09169a8d7fcfaa680f26277a4640bff1, disabling compactions & flushes 2023-03-08 11:56:34,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:56:34,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:56:34,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. after waiting 0 ms 2023-03-08 11:56:34,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:56:34,277 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/.tmp/rep_barrier/3eab8cf2c2a34e3f985f3efff40affe0 2023-03-08 11:56:34,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/09169a8d7fcfaa680f26277a4640bff1/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-08 11:56:34,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:56:34,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 09169a8d7fcfaa680f26277a4640bff1: 2023-03-08 11:56:34,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678276418584.09169a8d7fcfaa680f26277a4640bff1. 2023-03-08 11:56:34,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 148f994b12b7c170e5fbb2fd36359918, disabling compactions & flushes 2023-03-08 11:56:34,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:56:34,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:56:34,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. after waiting 0 ms 2023-03-08 11:56:34,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:56:34,281 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3eab8cf2c2a34e3f985f3efff40affe0 2023-03-08 11:56:34,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithMonthlyPolicy/148f994b12b7c170e5fbb2fd36359918/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-08 11:56:34,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:56:34,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 148f994b12b7c170e5fbb2fd36359918: 2023-03-08 11:56:34,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1678276432026.148f994b12b7c170e5fbb2fd36359918. 2023-03-08 11:56:34,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e53083d9a161a7082c38a975fc8a0192, disabling compactions & flushes 2023-03-08 11:56:34,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:34,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:34,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. after waiting 0 ms 2023-03-08 11:56:34,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:34,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMinorCompactionWithMonthlyPolicy/e53083d9a161a7082c38a975fc8a0192/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-08 11:56:34,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:34,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e53083d9a161a7082c38a975fc8a0192: 2023-03-08 11:56:34,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1678276559063.e53083d9a161a7082c38a975fc8a0192. 2023-03-08 11:56:34,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b2090f00e0a2d838e28358e6d6297e4, disabling compactions & flushes 2023-03-08 11:56:34,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:56:34,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:56:34,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. after waiting 0 ms 2023-03-08 11:56:34,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:56:34,291 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/1b2090f00e0a2d838e28358e6d6297e4/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 11:56:34,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:56:34,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b2090f00e0a2d838e28358e6d6297e4: 2023-03-08 11:56:34,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1678276512494.1b2090f00e0a2d838e28358e6d6297e4. 2023-03-08 11:56:34,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b617fde530c0c1bebf3fa2fa6eab77bb, disabling compactions & flushes 2023-03-08 11:56:34,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:56:34,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:56:34,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. after waiting 0 ms 2023-03-08 11:56:34,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:56:34,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/b617fde530c0c1bebf3fa2fa6eab77bb/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 11:56:34,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:56:34,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b617fde530c0c1bebf3fa2fa6eab77bb: 2023-03-08 11:56:34,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1678276539566.b617fde530c0c1bebf3fa2fa6eab77bb. 2023-03-08 11:56:34,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8fd6f903c75ee810ecad794fff5abb6e, disabling compactions & flushes 2023-03-08 11:56:34,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:56:34,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:56:34,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. after waiting 0 ms 2023-03-08 11:56:34,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:56:34,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/8fd6f903c75ee810ecad794fff5abb6e/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-08 11:56:34,299 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:56:34,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8fd6f903c75ee810ecad794fff5abb6e: 2023-03-08 11:56:34,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1678276457225.8fd6f903c75ee810ecad794fff5abb6e. 2023-03-08 11:56:34,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ff22c265b3d4c9370e800d08d19c30b0, disabling compactions & flushes 2023-03-08 11:56:34,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:34,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:34,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. after waiting 0 ms 2023-03-08 11:56:34,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:34,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ff22c265b3d4c9370e800d08d19c30b0/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 11:56:34,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:34,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ff22c265b3d4c9370e800d08d19c30b0: 2023-03-08 11:56:34,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1678276572400.ff22c265b3d4c9370e800d08d19c30b0. 2023-03-08 11:56:34,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 836c4c9f42bf977b9123d7bbe0300d0d, disabling compactions & flushes 2023-03-08 11:56:34,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:56:34,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:56:34,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. after waiting 0 ms 2023-03-08 11:56:34,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:56:34,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/836c4c9f42bf977b9123d7bbe0300d0d/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-08 11:56:34,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:56:34,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 836c4c9f42bf977b9123d7bbe0300d0d: 2023-03-08 11:56:34,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678276475101.836c4c9f42bf977b9123d7bbe0300d0d. 2023-03-08 11:56:34,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 91d8ee5ab3b818ebe215b0c4873b5950, disabling compactions & flushes 2023-03-08 11:56:34,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:56:34,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:56:34,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. after waiting 0 ms 2023-03-08 11:56:34,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:56:34,308 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testScannerOnBulkLoadRefHFiles/91d8ee5ab3b818ebe215b0c4873b5950/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-08 11:56:34,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:56:34,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 91d8ee5ab3b818ebe215b0c4873b5950: 2023-03-08 11:56:34,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1678276457225.91d8ee5ab3b818ebe215b0c4873b5950. 2023-03-08 11:56:34,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bb0058b62005e791ee17613797cadc96, disabling compactions & flushes 2023-03-08 11:56:34,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:56:34,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:56:34,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. after waiting 0 ms 2023-03-08 11:56:34,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:56:34,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/bb0058b62005e791ee17613797cadc96/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 11:56:34,312 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:56:34,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bb0058b62005e791ee17613797cadc96: 2023-03-08 11:56:34,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1678276512494.bb0058b62005e791ee17613797cadc96. 2023-03-08 11:56:34,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ffda891a90e48be952988d132e357ad3, disabling compactions & flushes 2023-03-08 11:56:34,313 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:34,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:34,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. after waiting 0 ms 2023-03-08 11:56:34,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:34,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/ffda891a90e48be952988d132e357ad3/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 11:56:34,316 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:34,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ffda891a90e48be952988d132e357ad3: 2023-03-08 11:56:34,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1678276572400.ffda891a90e48be952988d132e357ad3. 2023-03-08 11:56:34,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f1ce88c61365d5c7b6e6a24b8a887540, disabling compactions & flushes 2023-03-08 11:56:34,316 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:56:34,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:56:34,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. after waiting 0 ms 2023-03-08 11:56:34,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:56:34,316 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing f1ce88c61365d5c7b6e6a24b8a887540 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-08 11:56:34,326 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-08 11:56:34,452 DEBUG [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1505): Waiting on 1588230740, 30ea0c03bb5d1a1afdaa8ef87fc4f610, 6249f9a64ffef0dbc4b217b148070179, 6334f697e0e777acd41789a8c25bc8c9, 8a2814671292e9b2760956bf02135606, f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:56:34,653 DEBUG [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1505): Waiting on 1588230740, 30ea0c03bb5d1a1afdaa8ef87fc4f610, 6249f9a64ffef0dbc4b217b148070179, 6334f697e0e777acd41789a8c25bc8c9, 8a2814671292e9b2760956bf02135606, f1ce88c61365d5c7b6e6a24b8a887540 2023-03-08 11:56:34,688 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/.tmp/table/f259a4deedb54f08bacd677525569fdb 2023-03-08 11:56:34,692 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f259a4deedb54f08bacd677525569fdb 2023-03-08 11:56:34,693 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/.tmp/info/c01256d098904c3c8d95bb504d713fef as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/info/c01256d098904c3c8d95bb504d713fef 2023-03-08 11:56:34,696 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c01256d098904c3c8d95bb504d713fef 2023-03-08 11:56:34,696 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/info/c01256d098904c3c8d95bb504d713fef, entries=226, sequenceid=130, filesize=37.9 K 2023-03-08 11:56:34,697 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/.tmp/rep_barrier/3eab8cf2c2a34e3f985f3efff40affe0 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/rep_barrier/3eab8cf2c2a34e3f985f3efff40affe0 2023-03-08 11:56:34,700 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3eab8cf2c2a34e3f985f3efff40affe0 2023-03-08 11:56:34,700 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/rep_barrier/3eab8cf2c2a34e3f985f3efff40affe0, entries=1, sequenceid=130, filesize=5.0 K 2023-03-08 11:56:34,701 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/.tmp/table/f259a4deedb54f08bacd677525569fdb as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/table/f259a4deedb54f08bacd677525569fdb 2023-03-08 11:56:34,704 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f259a4deedb54f08bacd677525569fdb 2023-03-08 11:56:34,704 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/table/f259a4deedb54f08bacd677525569fdb, entries=27, sequenceid=130, filesize=6.6 K 2023-03-08 11:56:34,705 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 453ms, sequenceid=130, compaction requested=false 2023-03-08 11:56:34,709 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-08 11:56:34,709 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-08 11:56:34,709 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-08 11:56:34,709 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-08 11:56:34,709 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-08 11:56:34,721 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540/.tmp/info/d670c14ec9f04c8798e59a495fe27986 2023-03-08 11:56:34,724 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540/.tmp/info/d670c14ec9f04c8798e59a495fe27986 as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540/info/d670c14ec9f04c8798e59a495fe27986 2023-03-08 11:56:34,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540/info/d670c14ec9f04c8798e59a495fe27986, entries=3, sequenceid=7, filesize=4.8 K 2023-03-08 11:56:34,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for f1ce88c61365d5c7b6e6a24b8a887540 in 412ms, sequenceid=7, compaction requested=false 2023-03-08 11:56:34,731 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/hbase/namespace/f1ce88c61365d5c7b6e6a24b8a887540/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-08 11:56:34,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:56:34,731 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f1ce88c61365d5c7b6e6a24b8a887540: 2023-03-08 11:56:34,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1678276416880.f1ce88c61365d5c7b6e6a24b8a887540. 2023-03-08 11:56:34,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6249f9a64ffef0dbc4b217b148070179, disabling compactions & flushes 2023-03-08 11:56:34,732 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:56:34,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:56:34,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. after waiting 0 ms 2023-03-08 11:56:34,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:56:34,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/ns/testMinorCompaction/6249f9a64ffef0dbc4b217b148070179/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 11:56:34,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:56:34,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6249f9a64ffef0dbc4b217b148070179: 2023-03-08 11:56:34,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1678276539566.6249f9a64ffef0dbc4b217b148070179. 2023-03-08 11:56:34,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6334f697e0e777acd41789a8c25bc8c9, disabling compactions & flushes 2023-03-08 11:56:34,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:56:34,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:56:34,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. after waiting 0 ms 2023-03-08 11:56:34,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:56:34,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionWithWeeklyPolicy/6334f697e0e777acd41789a8c25bc8c9/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-08 11:56:34,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:56:34,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6334f697e0e777acd41789a8c25bc8c9: 2023-03-08 11:56:34,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1678276462226.6334f697e0e777acd41789a8c25bc8c9. 2023-03-08 11:56:34,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8a2814671292e9b2760956bf02135606, disabling compactions & flushes 2023-03-08 11:56:34,738 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:56:34,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:56:34,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. after waiting 0 ms 2023-03-08 11:56:34,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:56:34,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testCompactionWithHFileLink/8a2814671292e9b2760956bf02135606/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 11:56:34,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:56:34,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8a2814671292e9b2760956bf02135606: 2023-03-08 11:56:34,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1678276512494.8a2814671292e9b2760956bf02135606. 2023-03-08 11:56:34,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 30ea0c03bb5d1a1afdaa8ef87fc4f610, disabling compactions & flushes 2023-03-08 11:56:34,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:34,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:34,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. after waiting 0 ms 2023-03-08 11:56:34,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:34,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/data/default/testMajorCompactionFromAdmin/30ea0c03bb5d1a1afdaa8ef87fc4f610/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 11:56:34,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:34,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 30ea0c03bb5d1a1afdaa8ef87fc4f610: 2023-03-08 11:56:34,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1678276572400.30ea0c03bb5d1a1afdaa8ef87fc4f610. 2023-03-08 11:56:34,853 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,45551,1678276414946; all regions closed. 2023-03-08 11:56:34,859 DEBUG [RS:0;jenkins-hbase4:45551] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/oldWALs 2023-03-08 11:56:34,859 INFO [RS:0;jenkins-hbase4:45551] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45551%2C1678276414946.meta:.meta(num 1678276416570) 2023-03-08 11:56:34,863 DEBUG [RS:0;jenkins-hbase4:45551] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/oldWALs 2023-03-08 11:56:34,863 INFO [RS:0;jenkins-hbase4:45551] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45551%2C1678276414946:(num 1678276416421) 2023-03-08 11:56:34,863 DEBUG [RS:0;jenkins-hbase4:45551] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:56:34,863 INFO [RS:0;jenkins-hbase4:45551] regionserver.LeaseManager(133): Closed leases 2023-03-08 11:56:34,863 INFO [RS:0;jenkins-hbase4:45551] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-08 11:56:34,863 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-08 11:56:34,864 INFO [RS:0;jenkins-hbase4:45551] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45551 2023-03-08 11:56:34,872 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45551,1678276414946 2023-03-08 11:56:34,872 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 11:56:34,872 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 11:56:34,872 ERROR [Listener at localhost/36211-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@73555566 rejected from java.util.concurrent.ThreadPoolExecutor@776acfd4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 11:56:34,874 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45551,1678276414946] 2023-03-08 11:56:34,874 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45551,1678276414946; numProcessing=1 2023-03-08 11:56:34,875 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,45551,1678276414946 already deleted, retry=false 2023-03-08 11:56:34,875 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45551,1678276414946 expired; onlineServers=0 2023-03-08 11:56:34,875 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,46487,1678276413741' ***** 2023-03-08 11:56:34,875 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-08 11:56:34,876 DEBUG [M:0;jenkins-hbase4:46487] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@377511f1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-08 11:56:34,876 INFO [M:0;jenkins-hbase4:46487] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,46487,1678276413741 2023-03-08 11:56:34,876 INFO [M:0;jenkins-hbase4:46487] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,46487,1678276413741; all regions closed. 2023-03-08 11:56:34,876 DEBUG [M:0;jenkins-hbase4:46487] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 11:56:34,876 DEBUG [M:0;jenkins-hbase4:46487] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-08 11:56:34,876 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-08 11:56:34,876 DEBUG [M:0;jenkins-hbase4:46487] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-08 11:56:34,876 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678276416071] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678276416071,5,FailOnTimeoutGroup] 2023-03-08 11:56:34,876 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678276416071] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678276416071,5,FailOnTimeoutGroup] 2023-03-08 11:56:34,876 INFO [M:0;jenkins-hbase4:46487] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-08 11:56:34,876 INFO [M:0;jenkins-hbase4:46487] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-08 11:56:34,876 INFO [M:0;jenkins-hbase4:46487] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-03-08 11:56:34,877 DEBUG [M:0;jenkins-hbase4:46487] master.HMaster(1512): Stopping service threads 2023-03-08 11:56:34,877 INFO [M:0;jenkins-hbase4:46487] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-08 11:56:34,877 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-08 11:56:34,877 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 11:56:34,877 ERROR [M:0;jenkins-hbase4:46487] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-08 11:56:34,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 11:56:34,878 INFO [M:0;jenkins-hbase4:46487] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-08 11:56:34,878 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-08 11:56:34,878 DEBUG [M:0;jenkins-hbase4:46487] zookeeper.ZKUtil(398): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-08 11:56:34,878 WARN [M:0;jenkins-hbase4:46487] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-08 11:56:34,878 INFO [M:0;jenkins-hbase4:46487] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-08 11:56:34,878 INFO [M:0;jenkins-hbase4:46487] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-08 11:56:34,878 DEBUG [M:0;jenkins-hbase4:46487] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-08 11:56:34,878 INFO [M:0;jenkins-hbase4:46487] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 11:56:34,878 DEBUG [M:0;jenkins-hbase4:46487] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 11:56:34,878 DEBUG [M:0;jenkins-hbase4:46487] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-08 11:56:34,878 DEBUG [M:0;jenkins-hbase4:46487] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 11:56:34,878 INFO [M:0;jenkins-hbase4:46487] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.47 KB heapSize=598.17 KB 2023-03-08 11:56:34,974 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 11:56:34,974 INFO [RS:0;jenkins-hbase4:45551] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,45551,1678276414946; zookeeper connection closed. 2023-03-08 11:56:34,974 ERROR [Listener at localhost/36211-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5a377a30 rejected from java.util.concurrent.ThreadPoolExecutor@776acfd4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 11:56:34,974 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): regionserver:45551-0x100c3f9af850001, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 11:56:34,974 ERROR [Listener at localhost/36211-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@222591bc rejected from java.util.concurrent.ThreadPoolExecutor@776acfd4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 11:56:34,974 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@497b4506] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@497b4506 2023-03-08 11:56:34,975 INFO [Listener at localhost/36211] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-08 11:56:35,286 INFO [M:0;jenkins-hbase4:46487] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.47 KB at sequenceid=1005 (bloomFilter=true), to=hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1513a602739244f8957748546a46c65c 2023-03-08 11:56:35,289 INFO [M:0;jenkins-hbase4:46487] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1513a602739244f8957748546a46c65c 2023-03-08 11:56:35,289 DEBUG [M:0;jenkins-hbase4:46487] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1513a602739244f8957748546a46c65c as hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1513a602739244f8957748546a46c65c 2023-03-08 11:56:35,292 INFO [M:0;jenkins-hbase4:46487] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1513a602739244f8957748546a46c65c 2023-03-08 11:56:35,292 INFO [M:0;jenkins-hbase4:46487] regionserver.HStore(1080): Added hdfs://localhost:33137/user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1513a602739244f8957748546a46c65c, entries=123, sequenceid=1005, filesize=33.3 K 2023-03-08 11:56:35,293 INFO [M:0;jenkins-hbase4:46487] regionserver.HRegion(2948): Finished flush of dataSize ~512.47 KB/524770, heapSize ~598.16 KB/612512, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1005, compaction requested=false 2023-03-08 11:56:35,294 INFO [M:0;jenkins-hbase4:46487] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 11:56:35,294 DEBUG [M:0;jenkins-hbase4:46487] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 11:56:35,297 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/38c39ffa-074c-ff86-9852-d16b100c9f1c/MasterData/WALs/jenkins-hbase4.apache.org,46487,1678276413741/jenkins-hbase4.apache.org%2C46487%2C1678276413741.1678276415400 not finished, retry = 0 2023-03-08 11:56:35,398 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-08 11:56:35,398 INFO [M:0;jenkins-hbase4:46487] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-08 11:56:35,399 INFO [M:0;jenkins-hbase4:46487] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46487 2023-03-08 11:56:35,403 DEBUG [M:0;jenkins-hbase4:46487] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,46487,1678276413741 already deleted, retry=false 2023-03-08 11:56:35,504 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 11:56:35,504 INFO [M:0;jenkins-hbase4:46487] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,46487,1678276413741; zookeeper connection closed. 2023-03-08 11:56:35,504 ERROR [Listener at localhost/36211-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2b43de04 rejected from java.util.concurrent.ThreadPoolExecutor@7dadce72[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 11:56:35,504 DEBUG [Listener at localhost/36211-EventThread] zookeeper.ZKWatcher(600): master:46487-0x100c3f9af850000, quorum=127.0.0.1:51863, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 11:56:35,505 ERROR [Listener at localhost/36211-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4062da31 rejected from java.util.concurrent.ThreadPoolExecutor@7dadce72[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 11:56:35,506 WARN [Listener at localhost/36211] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-08 11:56:35,530 INFO [Listener at localhost/36211] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-08 11:56:35,637 WARN [BP-547615631-172.31.14.131-1678276410337 heartbeating to localhost/127.0.0.1:33137] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-08 11:56:35,637 WARN [BP-547615631-172.31.14.131-1678276410337 heartbeating to localhost/127.0.0.1:33137] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-547615631-172.31.14.131-1678276410337 (Datanode Uuid 8ded10a7-69e0-452e-8b64-90e5bfab4eb6) service to localhost/127.0.0.1:33137 2023-03-08 11:56:35,640 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/cluster_7d11bbd1-2ad2-65f1-e179-3d37987b50d1/dfs/data/data1/current/BP-547615631-172.31.14.131-1678276410337] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-08 11:56:35,640 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/805ec646-6443-dfab-69e3-17ce7c9069c8/cluster_7d11bbd1-2ad2-65f1-e179-3d37987b50d1/dfs/data/data2/current/BP-547615631-172.31.14.131-1678276410337] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-08 11:56:35,670 INFO [Listener at localhost/36211] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-08 11:56:35,807 INFO [Listener at localhost/36211] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-08 11:56:35,887 INFO [Listener at localhost/36211] hbase.HBaseTestingUtility(1293): Minicluster is down